bluuebunny/crossref_metadata_2025 huggingface dataset:
Treatment of lymphatic malformations in children: 10 years of experience
Authors:   Bogomolets National Medical University, Kyiv, Ukraine, Benzar I.M. ,   National Children’s Specialized Hospital «OKHMATDYT», Kyiv, Ukraine, Levytskyi A.F. ,   Bogomolets National Medical University, Kyiv, Ukraine,   National Children’s Specialized Hospital «OKHMATDYT», Kyiv, Ukraine, Diehtiarova D.S. ,   Bogomolets National Medical University, Kyiv, Ukraine,   National Children’s Specialized Hospital «OKHMATDYT», Kyiv, Ukraine, Godik O.S. ,   Bogomolets National Medical University, Kyiv, Ukraine,   National Children’s Specialized Hospital «OKHMATDYT», Kyiv, Ukraine, Dubrovin O.G. ,   Bogomolets National Medical University, Kyiv, Ukraine,   National Children’s Specialized Hospital «OKHMATDYT», Kyiv, Ukraine
Published: September 2022

Purpose is to identify the factors that determine the clinical features of the LMs, their complications and the choice of treatment options. Materials and methods. 225 children with LMs who underwent treatment since 2011 to 2020 were enrolled. They were grouped according to ISSVA 2018 classification. Clinical presentation, treatment options, complications and outcomes were analyzed. One-variant analysis using Pearson’s χ2 test was used to evaluate qualitative data. Results. 214 (95.1%) patients had cystic LMs, 11 (4.9%) had combined LMs. 129 (57.3%) LMs were cervicofacial, 17 (11.9%) patients developed airways compression and 6 (4.2%) required tracheostomy. 8 (3.6%) patients developed 10 lymphatic leaks episodes, 3 were lethal. 112 (86.8%) patients with cervicofacial, 5 (11.6%) abdominal, 10 (84.6%) axillar, and 12 (92.2%) extremities LMs underwent sclerotherapy. Surgical resections prevailed in abdominal LMs, 32 (74.5%), versus 4 (3.1%) cervicofacial LMs. 10 (4.4%) patients with combined and complicated cystic LMs underwent systemic sirolimus therapy. Results of treatment were excellent in 116 (51.6%) patients, good in 82 (36.4%), satisfactory in 23 (10.2%), and unsatisfactory in 4 (1.8%), with 4 mortalities caused by sepsis (n=1) and by massive lymph leakage (n=3). Conclusions. The most life-threatening LMs complications are airways compression and uncontrollable lymphatic leakages. Several approaches were used, including surgery, sclerotherapy, and target therapy. Surgery is optional for abdominal LMs but potentially harmful in cervicofacial. Treatment of combined LM is continuous and requires a combination of different methods. The research was carried out in accordance with the principles of the Helsinki Declaration. The study protocol was approved by the Local Ethics Committee of all participating institutions. The informed consent of the patient was obtained for conducting the studies. No conflict of interests was declared by the authors. Keywords: lymphatic malformations, children, sclerotherapy, target therapy.
Simultaneous minimally invasive interventions in children with stones in the urinary tract
Authors:   Danylo Halytsky Lviv National Medical University, Ukraine, Nakonechnyy R.A. , Borzhievskyi A.T. ,   Danylo Halytsky Lviv National Medical University, Ukraine, Nakonechnyi Y.A. ,   Danylo Halytsky Lviv National Medical University, Ukraine, Nakonechnyi A.Y. ,   Danylo Halytsky Lviv National Medical University, Ukraine
Published: September 2022

Purpose - to substantiate the expediency of simultaneous minimally invasive interventions in children with urolithiasis. Materials and methods. The clinical material covers 7 children aged 5 to 14 years with stones in any part of the pelvicalyceal system and in the lower cystoid of the ipsilateral ureter without congenital anomalies of the urinary tract, which cause urodynamic disorders. In this patients group, the total stones size of the hollow renal system in the longest projection was more than 8 mm and in the lower third of the ureter 5-7 mm, and the stones density - from 750 HU (Hounsfield Units) and above. Results. In patients with urolithiasis, it is necessary to restore urodynamics as quickly as possible, removing all stones from the urinary tract, regardless of their number and location. This is especially important in children, because in this age group the kidney can develop and function only in favorable anatomical and physiological conditions. The problem is exacerbated by the combined stones location in different topical urinary tract areas, in particular, in the ureter and the pelvicalyceal system of the kidney. Simultaneous minimally invasive interventions with the use of contact lithotripsy - percutaneous nephrolithotripsy and ureteroscopy are the most optimal for its solution. Conclusions. Minimally invasive technologies in the treatment of urolithiasis in children are quite safe and allow to perform simultaneous surgery with the contemporaneous removal of all stones from various organs of the urinary system. Preventive drainage of the pelvicalyceal system with a JJ-stent minimizes the clinic of renal colic and creates positive conditions for ureteroscopy. The research was carried out in accordance with the principles of the Helsinki Declaration. The study protocol was approved by the Local Ethics Committee of the listed institution. Informed consent of children and their parents was obtained for the research. No conflict of interests was declared by the authors. Keywords: urolithiasis, ureteroscopy, percutaneous nephrolithotripsy, contact lithotripsy, calculus, children.
Approaches to surgical treatment of deep burns of the hand burdened with subfascial hypertension syndrome
Authors:   Danylo Halytsky Lviv National Medical University, Ukraine, Chemerys O.M. , Khimich S.D. ,   National Pirogov Memorial Medical University, Vinnitsa, Ukraine, Savchyn V.S. ,   Danylo Halytsky Lviv National Medical University, Ukraine
Published: September 2022

The problem of treatment of subfascial hypertension syndrome in burn injury of the hand remains relevant due to lack of vigilance about its possible development. Purpose - to improve the results of treatment of deep burns of the hand by using the tactics of early surgery. Materials and methods. The clinical material consisted of 195 patients with burn lesions of the hand. The main group consisted of 83 victims, who used the tactics of active early surgery, followed by recovery and reconstruction. The comparison group consisted of 112 patients - in the treatment of which the tactics of gradual necrectomies were used after the formation of demarcation lines with the subsequent healing of wounds with the use of ointment dressings. Results. The data obtained showed that timely diagnosis, which consisted of the right treatment tactics, reduced the number of amputations and prevented the disability of victims with burns. This was achieved through timely and technically correct method of performing open decompression fasciotomies, which managed to reduce the subfascial pressure in each affected case, which prevented critical ischemia of muscle fibers and the development of Folkman’s contracture. It is extremely important to perform open fasciotomies in the syndrome of subfascial hypertension, because only in this way a complete decompression of tissues was achieved, and a timely verified diagnosis ensured a good treatment result. Early active surgical tactics for deep burns became a priority, compared to industry standards, which was expressed in an increase in the number of early surgical interventions (54.2% vs 23.2%; p<0.02), which reduced the proportion of wound infectious complications (4.8% vs 18.2; p<0.01), reduce the number of cicatricial contractures (14.5% vs 50.9%; р<0.02). Conclusions. Timely diagnosis of subfascial hypertension of the hand, active early surgical tactics, the proposed technique of decompression fasciotomies significantly improved the results of treatment, which manifested itself in the restoration of motor function of the hand. The research was carried out in accordance with the principles of the Helsinki Declaration. The study protocol was approved by the Local Ethics Committee of all participating institutions. The informed consent of the patient was obtained for conducting the studies. No conflict of interests was declared by the authors. Keywords: burns, hand’s burns, subfascial hypertension syndrome, open decompression fasciotomies, wounds.
Anais do XXII Congresso Nacional de Engenharia de Segurança no Trabalho - CONEST
Authors: Barreto Clarice
Published: September 2022

A Associação Nacional de Engenharia de Segurança do Trabalho (ANEST) em conjunto com a Associação dos Engenheiros de Segurança do Trabalho do Mato Grosso (AMAEST), realizou nos dias 04, 05 e 06 de novembro de 2020, o “22º CONEST – Congresso Nacional de Engenharia de Segurança do Trabalho”. Evento Técnico-científico destinado a atualização e troca de experiências, abordando como tema central: INOVAÇÃO TECNOLÓGICA E O FUTURO DA ENGENHARIA DE SEGURANÇA DO TRABALHO, com a participação de renomados palestrantes nacionais e internacionais. Os diversos temas selecionados foram abordados em conferências, painéis e mini- cursos, utilizando-se de protocolos restritos e medidas preventivas rigorosas de saúde e segurança, tendo em vista o estado pandêmico em COVID-19 que o mundo se encontra.
Explosive Zeiten : eine deutsche Erfolgsgeschichte
Authors: Steinicke Horst
Published: September 2022

Von der Vision zur industriellen Realität - Sprengstoff formt LKW-Achsbrücken.
From genes to ecosystems: a synthesis of amphibian biodiversity research in Brazil
Authors: Haddad Célio F.B. , Lopes Carla M. , Becker C. Guilherme , da Silva Fernando R. , Lyra Mariana L.
Published: September 2022

Abstract Here, we summarize examples of significant advances in amphibian research supported by the São Paulo Research Foundation (FAPESP), focusing on recent discoveries in the fields of community ecology, habitat change, infection diseases, and multipurpose DNA sequencing. We demonstrated that FAPESP has been fundamental not only by directly funding research projects and scholarships, but also through its science training policy, fostering international collaborations with world-class research institutions, improving and consolidating new lines of research that often depended on a synergetic combination of different knowledge and complex tools. We emphasized that future studies will continue to focus on basic questions, such as description of new species, as well as taxonomic and systematic corrections. Furthermore, we also expect that there will be a strong integration among different disciplines using novel bioinformatics tools and modeling approaches, such as machine learning. These new approaches will be critical to further develop our understanding of foundational questions of amphibian life-history trait variation, disease transmission, community assembly, biogeography, and population forecasts under different global change scenarios such as agricultural expansion, agrochemical use, habitat loss, and climate change.
Optimal Kidney Exchange with Immunosuppressants
Authors: Aziz Haris , Cseh Ágnes , Dickerson John P. , McElfresh Duncan C.
Published: September 2022

Algorithms for exchange of kidneys is one of the key successful applications in market design, artificial intelligence, and operations research. Potent immunosuppressant drugs suppress the body's ability to reject a transplanted organ up to the point that a transplant across blood- or tissue-type incompatibility becomes possible. In contrast to the standard kidney exchange problem, we consider a setting that also involves the decision about which recipients receive from the limited supply of immunosuppressants that make them compatible with originally incompatible kidneys. We firstly present a general computational framework to model this problem. Our main contribution is a range of efficient algorithms that provide flexibility in terms of meeting meaningful objectives. Motivated by the current reality of kidney exchanges using sophisticated mathematical-programming-based clearing algorithms, we then present a general but scalable approach to optimal clearing with immunosuppression; we validate our approach on realistic data from a large fielded exchange.
ECG ODE-GAN: Learning Ordinary Differential Equations of ECG Dynamics via Generative Adversarial Learning
Authors: Golany Tomer , Freedman Daniel , Radinsky Kira
Published: September 2022

Understanding the dynamics of complex biological and physiological systems has been explored for many years in the form of physically-based mathematical simulators. The behavior of a physical system is often described via ordinary differential equations (ODE), referred to as the dynamics. In the standard case, the dynamics are derived from purely physical considerations. By contrast, in this work we study how the dynamics can be learned by a generative adversarial network which combines both physical and data considerations. As a use case, we focus on the dynamics of the heart signal electrocardiogram (ECG). We begin by introducing a new GAN framework, dubbed ODE-GAN, in which the generator learns the dynamics of a physical system in the form of an ordinary differential equation. Specifically, the generator network receives as input a value at a specific time step, and produces the derivative of the system at that time step. Thus, the ODE-GAN learns purely data-driven dynamics. We then show how to incorporate physical considerations into ODE-GAN. We achieve this through the introduction of an additional input to the ODE-GAN generator: physical parameters, which partially characterize the signal of interest. As we focus on ECG signals, we refer to this new framework as ECG-ODE-GAN. We perform an empirical evaluation and show that generating ECG heartbeats from our learned dynamics improves ECG heartbeat classification.
Hierarchical Graph Convolution Network for Traffic Forecasting
Authors: Guo Kan , Hu Yongli , Sun Yanfeng , Qian Sean , Gao Junbin , Yin Baocai
Published: September 2022

Traffic forecasting is attracting considerable interest due to its widespread application in intelligent transportation systems. Given the complex and dynamic traffic data, many methods focus on how to establish a spatial-temporal model to express the non-stationary traffic patterns. Recently, the latest Graph Convolution Network (GCN) has been introduced to learn spatial features while the time neural networks are used to learn temporal features. These GCN based methods obtain state-of-the-art performance. However, the current GCN based methods ignore the natural hierarchical structure of traffic systems which is composed of the micro layers of road networks and the macro layers of region networks, in which the nodes are obtained through pooling method and could include some hot traffic regions such as downtown and CBD etc., while the current GCN is only applied on the micro graph of road networks. In this paper, we propose a novel Hierarchical Graph Convolution Networks (HGCN) for traffic forecasting by operating on both the micro and macro traffic graphs. The proposed method is evaluated on two complex city traffic speed datasets. Compared to the latest GCN based methods like Graph WaveNet, the proposed HGCN gets higher traffic forecasting precision with lower computational cost.The website of the code is https://github.com/guokan987/HGCN.git.
Modeling the Compatibility of Stem Tracks to Generate Music Mashups
Authors: Huang Jiawen , Wang Ju-Chiang , Smith Jordan B. L. , Song Xuchen , Wang Yuxuan
Published: September 2022

A music mashup combines audio elements from two or more songs to create a new work. To reduce the time and effort required to make them, researchers have developed algorithms that predict the compatibility of audio elements. Prior work has focused on mixing unaltered excerpts, but advances in source separation enable the creation of mashups from isolated stems (e.g., vocals, drums, bass, etc.). In this work, we take advantage of separated stems not just for creating mashups, but for training a model that predicts the mutual compatibility of groups of excerpts, using self-supervised and semi-supervised methods. Specifically, we first produce a random mashup creation pipeline that combines stem tracks obtained via source separation, with key and tempo automatically adjusted to match, since these are prerequisites for high-quality mashups. To train a model to predict compatibility, we use stem tracks obtained from the same song as positive examples, and random combinations of stems with key and/or tempo unadjusted as negative examples. To improve the model and use more data, we also train on "average" examples: random combinations with matching key and tempo, where we treat them as unlabeled data as their true compatibility is unknown. To determine whether the combined signal or the set of stem signals is more indicative of the quality of the result, we experiment on two model architectures and train them using semi-supervised learning technique. Finally, we conduct objective and subjective evaluations of the system, comparing them to a standard rule-based system.
Deep Contextual Clinical Prediction with Reverse Distillation
Authors: Kodialam Rohan , Boiarsky Rebecca , Lim Justin , Sai Aditya , Dixit Neil , Sontag David
Published: September 2022

Healthcare providers are increasingly using machine learning to predict patient outcomes to make meaningful interventions. However, despite innovations in this area, deep learning models often struggle to match performance of shallow linear models in predicting these outcomes, making it difficult to leverage such techniques in practice. In this work, motivated by the task of clinical prediction from insurance claims, we present a new technique called reverse distillation which pretrains deep models by using high-performing linear models for initialization. We make use of the longitudinal structure of insurance claims datasets to develop Self Attention with Reverse Distillation, or SARD, an architecture that utilizes a combination of contextual embedding, temporal embedding and self-attention mechanisms and most critically is trained via reverse distillation. SARD outperforms state-of-the-art methods on multiple clinical prediction outcomes, with ablation studies revealing that reverse distillation is a primary driver of these improvements. Code is available at https://github.com/clinicalml/omop-learn.
Relational Classification of Biological Cells in Microscopy Images
Authors: Liu Ping , Bilgic Mustafa
Published: September 2022

We investigate the relational classification of biological cells in 2D microscopy images. Rather than treating each cell image independently, we investigate whether and how the neighborhood information of a cell can be informative for its prediction.
We propose a Relational Long Short-Term Memory (R-LSTM) algorithm, coupled with auto-encoders and convolutional neural networks, that can learn from both annotated and unlabeled microscopy images and that can utilize both the local and neighborhood information to perform an improved classification of biological cells. Experimental results on both synthetic and real datasets show that R-LSTM performs comparable to or better than six baselines.
Low-Rank Registration Based Manifolds for Convection-Dominated PDEs
Authors: Mojgani Rambod , Balajewicz Maciej
Published: September 2022

We develop an auto-encoder-type nonlinear dimensionality reduction algorithm to enable the construction of reduced order models of systems governed by convection-dominated nonlinear partial differential equations (PDEs), i.e. snapshots of solutions with large Kolmogorov n-width. Although several existing nonlinear manifold learning methods, such as LLE, ISOMAP, MDS, etc., appear as compelling candidates to reduce the dimensionality of such data, most are not applicable to reduced order modeling of PDEs, because: (i) they typically lack a straightforward mapping from the latent space to the high-dimensional physical space, and (ii) the identified latent variables are often difficult to interpret. In our proposed method, these limitations are overcome by training a low-rank diffeomorphic spatio-temporal grid that registers the output sequence of the PDEs on a non-uniform parameter/time-varying grid, such that the Kolmogorov n-width of the mapped data on the learned grid is minimized. We demonstrate the efficacy and interpretability of our proposed approach on several challenging manufactured computer vision-inspired tasks and physical systems.
Pragmatic Code Autocomplete
Authors: Poesia Gabriel , Goodman Noah
Published: September 2022

Human language is ambiguous, with intended meanings recovered via pragmatic reasoning in context. Such reliance on context is essential for the efficiency of human communication. Programming languages, in stark contrast, are defined by unambiguous grammars. In this work, we aim to make programming languages more concise by allowing programmers to utilize a controlled level of ambiguity. Specifically, we allow single-character abbreviations for common keywords and identifiers. Our system first proposes a set of strings that can be abbreviated by the user. Using only 100 abbreviations, we observe that a large dataset of Python code can be compressed by 15%, a number that can be improved even further by specializing the abbreviations to a particular code base. We then use a contextualized sequence-to-sequence model to rank potential expansions of inputs that include abbreviations. In an offline reconstruction task our model achieves accuracies ranging from 93% to 99%, depending on the programming language and user settings. The model is small enough to run on a commodity CPU in real-time. We evaluate the usability of our system in a user study, integrating it in Microsoft VSCode, a popular code text editor. We observe that our system performs well and is complementary to traditional autocomplete features.
StatEcoNet: Statistical Ecology Neural Networks for Species Distribution Modeling
Authors: Seo Eugene , Hutchinson Rebecca A. , Fu Xiao , Li Chelsea , Hallman Tyler A. , Kilbride John , Robinson W. Douglas
Published: September 2022

This paper focuses on a core task in computational sustainability and statistical ecology: species distribution modeling (SDM). In SDM, the occurrence pattern of a species on a landscape is predicted by environmental features based on observations at a set of locations. At first, SDM may appear to be a binary classification problem, and one might be inclined to employ classic tools (e.g., logistic regression, support vector machines, neural networks) to tackle it. However, wildlife surveys introduce structured noise (especially under-counting) in the species observations. If unaccounted for, these observation errors systematically bias SDMs. To address the unique challenges of SDM, this paper proposes a framework called StatEcoNet. Specifically, this work employs a graphical generative model in statistical ecology to serve as the skeleton of the proposed computational framework and carefully integrates neural networks under the framework. The advantages of StatEcoNet over related approaches are demonstrated on simulated datasets as well as bird species data. Since SDMs are critical tools for ecological science and natural resource management, StatEcoNet may offer boosted computational and analytical powers to a wide range of applications that have significant social impacts, e.g., the study and conservation of threatened species.
A Spatial Regulated Patch-Wise Approach for Cervical Dysplasia Diagnosis
Authors: Zhang Ying , Yin Yifang , Liu Zhenguang , Zimmermann Roger
Published: September 2022

Cervical dysplasia diagnosis via visual investigation is a challenging problem. Recent approaches use deep learning techniques to extract features and require the downsampling of high-resolution cervical screening images to smaller sizes for training. Such a reduction may result in the loss of visual details that appear weakly and locally within a cervical image. To overcome this challenge, our work divides an image into patches and then represents it from patch features. We aggregate patch patterns into an image feature in a weighted manner by considering the patch--image label relation. The weights are visualized as a heatmap to explain where the diagnosis results come from. We further introduce a spatial regulator to guide the classifier to focus on the cervix region and to adjust the weight distribution, without requiring any manual annotations of the cervix region. A novel iterative algorithm is designed to refine the regulator, which is able to capture the variations in cervix center locations and shapes. Experiments on an 18-year real-world dataset indicate a minimal of 3.47%, 4.59%, 8.54% improvements over the state-of-the-art in accuracy, F1, and recall measures, respectively.
Visual Relation Detection using Hybrid Analogical Learning
Authors: Chen Kezhen , Forbus Ken
Published: September 2022

Visual Relation Detection is currently one of the most popular problems for visual understanding. Many deep-learning models are designed for relation detection on images and have achieved impressive results. However, deep-learning models have several serious problems, including poor training-efficiency and lack of understandability. Psychologists have ample evidence that analogy is central in human learning and reasoning, including visual reasoning. This paper introduces a new hybrid system for visual relation detection combining deep-learning models and analogical generalization. Object bounding boxes and masks are detected using deep-learning models and analogical generalization over qualitative representations is used for visual relation detection between object pairs. Experiments on the Visual Relation Detection dataset indicates that our hybrid system gets comparable results on the task and is more training-efficient and explainable than pure deep-learning models.
PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception
Authors: Netanyahu Aviv , Shu Tianmin , Katz Boris , Barbu Andrei , Tenenbaum Joshua B.
Published: September 2022

The ability to perceive and reason about social interactions in the context of physical environments is core to human social intelligence and human-machine cooperation. However, no prior dataset or benchmark has systematically evaluated physically grounded perception of complex social interactions that go beyond short actions, such as high-fiving, or simple group activities, such as gathering. In this work, we create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions by including social concepts such as helping another agent. PHASE consists of 2D animations of pairs of agents moving in a continuous space generated procedurally using a physics engine and a hierarchical planner. Agents have a limited field of view, and can interact with multiple objects, in an environment that has multiple landmarks and obstacles. Using PHASE, we design a social recognition task and a social prediction task. PHASE is validated with human experiments demonstrating that humans perceive rich interactions in the social events, and that the simulated agents behave similarly to humans. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE (SIMulation, Planning and Local Estimation), which outperforms state-of-the-art feed-forward neural networks. We hope that PHASE can serve as a difficult new challenge for developing new models that can recognize complex social interactions.
Multi-View Representation Learning with Manifold Smoothness
Authors: Li Shu , Wang Wei , Li Wen-Tao , Chen Pan
Published: September 2022

Multi-view representation learning attempts to learn a representation from multiple views and most existing methods are unsupervised. However, representation learned only from unlabeled data may not be discriminative enough for further applications (e.g., clustering and classification). For this reason, semi-supervised methods which could use unlabeled data along with the labeled data for multi-view representation learning need to be developed. Manifold information plays an important role in semi-supervised learning, but it has not been considered for multi-view representation learning. In this paper, we introduce the manifold smoothness into multi-view representation learning and propose MvDGAT which learns the representation and the intrinsic manifold simultaneously with graph attention network. Experiments conducted on real-world datasets reveal that our MvDGAT can achieve better performance than state-of-the-art methods.
Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network
Authors: Li Yehao , Pan Yingwei , Yao Ting , Chen Jingwen , Mei Tao
Published: September 2022

Despite having impressive vision-language (VL) pretraining with BERT-based encoder for VL understanding, the pretraining of a universal encoder-decoder for both VL understanding and generation remains challenging. The difficulty originates from the inherently different peculiarities of the two disciplines, e.g., VL understanding tasks capitalize on the unrestricted message passing across modalities, while generation tasks only employ visual-to-textual message passing. In this paper, we start with a two-stream decoupled design of encoder-decoder structure, in which two decoupled cross-modal encoder and decoder are involved to separately perform each type of proxy tasks, for simultaneous VL understanding and generation pretraining. Moreover, for VL pretraining, the dominant way is to replace some input visual/word tokens with mask tokens and enforce the multi-modal encoder/decoder to reconstruct the original tokens, but no mask token is involved when fine-tuning on downstream tasks. As an alternative, we propose a primary scheduled sampling strategy that elegantly mitigates such discrepancy via pretraining encoder-decoder in a two-pass manner. Extensive experiments demonstrate the compelling generalizability of our pretrained encoder-decoder by fine-tuning on four VL understanding and generation downstream tasks. Source code is available at https://github.com/YehLi/TDEN.
Large Norms of CNN Layers Do Not Hurt Adversarial Robustness
Authors: Liang Youwei , Huang Dong
Published: September 2022

Since the Lipschitz properties of convolutional neural networks (CNNs) are widely considered to be related to adversarial robustness, we theoretically characterize the L-1 norm and L-infinity norm of 2D multi-channel convolutional layers and provide efficient methods to compute the exact L-1 norm and L-infinity norm. Based on our theorem, we propose a novel regularization method termed norm decay, which can effectively reduce the norms of convolutional layers and fully-connected layers. Experiments show that norm-regularization methods, including norm decay, weight decay, and singular value clipping, can improve generalization of CNNs. However, they can slightly hurt adversarial robustness. Observing this unexpected phenomenon, we compute the norms of layers in the CNNs trained with three different adversarial training frameworks and surprisingly find that adversarially robust CNNs have comparable or even larger layer norms than their non-adversarially robust counterparts. Furthermore, we prove that under a mild assumption, adversarially robust classifiers can be achieved using neural networks, and an adversarially robust neural network can have an arbitrarily large Lipschitz constant. For this reason, enforcing small norms on CNN layers may be neither necessary nor effective in achieving adversarial robustness. The code is available at https://github.com/youweiliang/norm_robustness.
From Label Smoothing to Label Relaxation
Authors: Lienen Julian , Hüllermeier Eyke
Published: September 2022

Regularization of (deep) learning models can be realized at the model, loss, or data level. As a technique somewhere in-between loss and data, label smoothing turns deterministic class labels into probability distributions, for example by uniformly distributing a certain part of the probability mass over all classes. A predictive model is then trained on these distributions as targets, using cross-entropy as loss function. While this method has shown improved performance compared to non-smoothed cross-entropy, we argue that the use of a smoothed though still precise probability distribution as a target can be questioned from a theoretical perspective. As an alternative, we propose a generalized technique called label relaxation, in which the target is a set of probabilities represented in terms of an upper probability distribution. This leads to a genuine relaxation of the target instead of a distortion, thereby reducing the risk of incorporating an undesirable bias in the learning process. Methodically, label relaxation leads to the minimization of a novel type of loss function, for which we propose a suitable closed-form expression for model optimization. The effectiveness of the approach is demonstrated in an empirical study on image data.
Auto-Encoding Transformations in Reparameterized Lie Groups for Unsupervised Learning
Authors: Lin Feng , Xu Haohang , Li Houqiang , Xiong Hongkai , Qi Guo-Jun
Published: September 2022

Unsupervised training of deep representations has demonstrated remarkable potentials in mitigating the prohibitive expenses on annotating labeled data recently. Among them is predicting transformations as a pretext task to self-train representations, which has shown great potentials for unsupervised learning. However, existing approaches in this category learn representations by either treating a discrete set of transformations as separate classes, or using the Euclidean distance as the metric to minimize the errors between transformations. None of them has been dedicated to revealing the vital role of the geometry of transformation groups in learning representations. Indeed, an image must continuously transform along the curved manifold of a transformation group rather than through a straight line in the forbidden ambient Euclidean space. This suggests the use of geodesic distance to minimize the errors between the estimated and groundtruth transformations. Particularly, we focus on homographies, a general group of planar transformations containing the Euclidean, similarity and affine transformations as its special cases. To avoid an explicit computing of intractable Riemannian logarithm, we project homographies onto an alternative group of rotation transformations SR(3) with a tractable form of geodesic distance. Experiments demonstrate the proposed approach to Auto-Encoding Transformations exhibits superior performances on a variety of recognition problems.
Train a One-Million-Way Instance Classifier for Unsupervised Visual Representation Learning
Authors: Liu Yu , Huang Lianghua , Pan Pan , Wang Bin , Xu Yinghui , Jin Rong
Published: September 2022

This paper presents a simple unsupervised visual representation learning method with a pretext task of discriminating all images in a dataset using a parametric, instance-level classifier. The overall framework is a replica of a supervised classification model, where semantic classes (e.g., dog, bird, and ship) are replaced by instance IDs. However, scaling up the classification task from thousands of semantic labels to millions of instance labels brings specific challenges including 1) the large-scale softmax computation; 2) the slow convergence due to the infrequent visiting of instance samples; and 3) the massive number of negative classes that can be noisy. This work presents several novel techniques to handle these difficulties. First, we introduce a hybrid parallel training framework to make large-scale training feasible. Second, we present a raw-feature initialization mechanism for classification weights, which we assume offers a contrastive prior for instance discrimination and can clearly speed up converge in our experiments. Finally, we propose to smooth the labels of a few hardest classes to avoid optimizing over very similar negative pairs. While being conceptually simple, our framework achieves competitive or superior performance compared to state-of-the-art unsupervised approaches, i.e., SimCLR, MoCoV2, and PIC under ImageNet linear evaluation protocol and on several downstream visual tasks, verifying that full instance classification is a strong pretraining technique for many semantic visual tasks.
Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate
Authors: Mutti Mirco , Pratissoli Lorenzo , Restelli Marcello
Published: September 2022

In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy? In this paper, we argue that the entropy of the state distribution induced by finite-horizon trajectories is a sensible target. Especially, we present a novel and practical policy-search algorithm, Maximum Entropy POLicy optimization (MEPOL), to learn a policy that maximizes a non-parametric, $k$-nearest neighbors estimate of the state distribution entropy. In contrast to known methods, MEPOL is completely model-free as it requires neither to estimate the state distribution of any policy nor to model transition dynamics. Then, we empirically show that MEPOL allows learning a maximum-entropy exploration policy in high-dimensional, continuous-control domains, and how this policy facilitates learning meaningful reward-based tasks downstream.
Differentially Private k-Means via Exponential Mechanism and Max Cover
Authors: Nguyen Huy L. , Chaturvedi Anamay , Xu Eric Z
Published: September 2022

We introduce a new (ϵₚ, δₚ)-differentially private algorithm for the k-means clustering problem. Given a dataset in Euclidean space, the k-means clustering problem requires one to find k points in that space such that the sum of squares of Euclidean distances between each data point and its closest respective point among the k returned is minimised. Although there exist privacy-preserving methods with good theoretical guarantees to solve this problem, in practice it is seen that it is the additive error which dictates the practical performance of these methods. By reducing the problem to a sequence of instances of maximum coverage on a grid, we are able to derive a new method that achieves lower additive error than previous works. For input datasets with cardinality n and diameter Δ, our algorithm has an O(Δ² (k log² n log(1/δₚ)/ϵₚ + k √(d log(1/δₚ))/ϵₚ)) additive error whilst maintaining constant multiplicative error. We conclude with some experiments and find an improvement over previously implemented work for this problem.
An Information-Theoretic Framework for Unifying Active Learning Problems
Authors: Nguyen Quoc Phong , Low Bryan Kian Hsiang , Jaillet Patrick
Published: September 2022

This paper presents an information-theoretic framework for unifying active learning problems: level set estimation (LSE), Bayesian optimization (BO), and their generalized variant. We first introduce a novel active learning criterion that subsumes an existing LSE algorithm and achieves state-of-the-art performance in LSE problems with a continuous input domain. Then, by exploiting the relationship between LSE and BO, we design a competitive information-theoretic acquisition function for BO that has interesting connections to upper confidence bound and max-value entropy search (MES). The latter connection reveals a drawback of MES which has important implications on not only MES but also on other MES-based acquisition functions. Finally, our unifying information-theoretic framework can be applied to solve a generalized problem of LSE and BO involving multiple level sets in a data-efficient manner. We empirically evaluate the performance of our proposed algorithms using synthetic benchmark functions, a real-world dataset, and in hyperparameter tuning of machine learning models.
Precision-based Boosting
Authors: Nikravan Mohammad Hossein , Movahedan Marjan , Zilles Sandra
Published: September 2022

AdaBoost is a highly popular ensemble classification method for which many variants have been published. This paper proposes a generic refinement of all of these AdaBoost variants. Instead of assigning weights based on the total error of the base classifiers (as in AdaBoost), our method uses class-specific error rates. On instance x it assigns a higher weight to a classifier predicting label y on x, if that classifier is less likely to make a mistake when it predicts class y. Like AdaBoost, our method is guaranteed to boost weak learners into strong learners. An empirical study on AdaBoost and one of its multi-class versions, SAMME, demonstrates the superiority of our method on datasets with more than 1,000 instances as well as on datasets with more than three classes.
Learning Deep Generative Models for Queuing Systems
Authors: Ojeda Cesar , Cvejoski Kostadin , Georgiev Bodgan , Bauckhage Christian , Schuecker Jannis , Sanchez Ramses J.
Published: September 2022

Modern society is heavily dependent on large scale client-server systems with applications ranging from Internet and Communication Services to sophisticated logistics and deployment of goods. To maintain and improve such a system, a careful study of client and server dynamics is needed – e.g. response/service times, aver-age number of clients at given times, etc. To this end, one traditionally relies, within the queuing theory formalism,on parametric analysis and explicit distribution forms.However, parametric forms limit the model’s expressiveness and could struggle on extensively large datasets. We propose a novel data-driven approach towards queuing systems: the Deep Generative Service Times. Our methodology delivers a flexible and scalable model for service and response times. We leverage the representation capabilities of Recurrent Marked Point Processes for the temporal dynamics of clients, as well as Wasserstein Generative Adversarial Network techniques, to learn deep generative models which are able to represent complex conditional service time distributions. We provide extensive experimental analysis on both empirical and synthetic datasets, showing the effectiveness of the proposed models
Vector Quantized Bayesian Neural Network Inference for Data Streams
Authors: Park Namuk , Lee Taekyu , Kim Songkuk
Published: September 2022

Bayesian neural networks (BNN) can estimate the uncertainty in predictions, as opposed to non-Bayesian neural networks (NNs). However, BNNs have been far less widely used than non-Bayesian NNs in practice since they need iterative NN executions to predict a result for one data, and it gives rise to prohibitive computational cost. This computational burden is a critical problem when processing data streams with low-latency. To address this problem, we propose a novel model VQ-BNN, which approximates BNN inference for data streams. In order to reduce the computational burden, VQ-BNN inference predicts NN only once and compensates the result with previously memorized predictions. To be specific, VQ-BNN inference for data streams is given by temporal exponential smoothing of recent predictions. The computational cost of this model is almost the same as that of non-Bayesian NNs. Experiments including semantic segmentation on real-world data show that this model performs significantly faster than BNNs while estimating predictive results comparable to or superior to the results of BNNs.
Fast Multi-view Discrete Clustering with Anchor Graphs
Authors: Qiang Qianyao , Zhang Bin , Wang Fei , Nie Feiping
Published: September 2022

Generally, the existing graph-based multi-view clustering models consists of two steps: (1) graph construction; (2) eigen-decomposition on the graph Laplacian matrix to compute a continuous cluster assignment matrix, followed by a post-processing algorithm to get the discrete one. However, both the graph construction and eigen-decomposition are time-consuming, and the two-stage process may deviate from directly solving the primal problem. To this end, we propose Fast Multi-view Discrete Clustering (FMDC) with anchor graphs, focusing on directly solving the spectral clustering problem with a small time cost. We efficiently generate representative anchors and construct anchor graphs on different views. The discrete cluster assignment matrix is directly obtained by performing clustering on the automatically aggregated graph. FMDC has a linear computational complexity with respect to the data scale, which is a significant improvement compared to the quadratic one. Extensive experiments on benchmark datasets demonstrate its efficiency and effectiveness.
Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection
Authors: Raff Edward , Fleshman William , Zak Richard , Anderson Hyrum S. , Filar Bobby , McLean Mark
Published: September 2022

Recent works within machine learning have been tackling inputs of ever increasing size, with cyber security presenting sequence classification problems of particularly extreme lengths. In the case of Windows executable malware detection, an input executable could be >=100 MB, which would translate to a time series with T=100,000,000 steps. To date, the closest approach to handling such task is MalConv --- a convolutional neural network capable of processing T=2,000,000 steps. Because the memory used by CNNs is O(T), this has prevented many from processing all executables or further extending the MalConv approach. In this work, we develop a new approach to temporal max pooling that makes the required memory invariant to the sequence length T. This makes MalConv 116x more memory efficient, and up to 25.8x faster to train, while removing the input length restrictions to MalConv. We re-invest these gains into improving the MalConv architecture by developing a new Global Channel Gating design, giving us an attention mechanism capable of learning feature interactions across 100 million time steps in an efficient manner, a capability lacked by the original MalConv approach.
A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization
Authors: Sankar Adepu Ravi , Khasbage Yash , Vigneswaran Rahul , N Balasubramanian Vineeth
Published: September 2022

Loss landscape analysis is extremely useful for a deeper understanding of the generalization ability of deep neural network models. In this work, we propose a layerwise loss landscape analysis where the loss surface at every layer is studied independently and also on how each correlates to the overall loss surface. We study the layerwise loss landscape by studying the eigenspectra of the Hessian at each layer. In particular, our results show that the layerwise Hessian geometry is largely similar to the entire Hessian. We also report an interesting phenomenon where the Hessian eigenspectrum of middle layers of the deep neural network are observed to most similar to the overall Hessian eigenspectrum. We also show that the maximum eigenvalue and the trace of the Hessian (both full network and layerwise) reduce as training of the network progresses. We leverage on these observations to propose a new regularizer based on the trace of the layerwise Hessian. Penalizing the trace of the Hessian at every layer indirectly forces Stochastic Gradient Descent to converge to flatter minima, which are shown to have better generalization performance. In particular, we show that such a layerwise regularizer can be leveraged to penalize the middlemost layers alone, which yields promising results. Our empirical studies on well-known deep nets across datasets support the claims of this work.