Miracle-dz/UniFAR-FTUs huggingface dataset:
When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks

Recent advances in adversarial attacks uncover the intrinsic vulnerability of modern deep neural networks. Since then, extensive efforts have been devoted to enhancing the robustness of deep networks via specialized learning algorithms and loss functions. In this work, we take an architectural perspective and investigate the patterns of network architectures that are resilient to adversarial attacks. To obtain the large number of networks needed for this study, we adopt one-shot neural architecture search, training a large network for once and then finetuning the sub-networks sampled therefrom. The sampled architectures together with the accuracies they achieve provide a rich basis for our study. Our ''robust architecture Odyssey'' reveals several valuable observations: 1) densely connected patterns result in improved robustness; 2) under computational budget, adding convolution operations to direct connection edge is effective; 3) flow of solution procedure (FSP) matrix is a good indicator of network robustness. Based on these observations, we discover a family of robust architectures (RobNets). On various datasets, including CIFAR, SVHN, Tiny-ImageNet, and ImageNet, RobNets exhibit superior robustness performance to other widely used architectures. Notably, RobNets substantially improve the robust accuracy (~5% absolute gains) under both white-box and black-box attacks, even with fewer parameter numbers. Code is available at https://github.com/gmh14/RobNets.
Biologically Plausible Sequence Learning with Spiking Neural Networks

Motivated by the celebrated discrete-time model of nervous activity outlined by McCulloch and Pitts in 1943, we propose a novel continuous-time model, the McCulloch-Pitts network (MPN), for sequence learning in spiking neural networks. Our model has a local learning rule, such that the synaptic weight updates depend only on the information directly accessible by the synapse. By exploiting asymmetry in the connections between binary neurons, we show that MPN can be trained to robustly memorize multiple spatiotemporal patterns of binary vectors, generalizing the ability of the symmetric Hopfield network to memorize static spatial patterns. In addition, we demonstrate that the model can efficiently learn sequences of binary pictures as well as generative models for experimental neural spike-train data. Our learning rule is consistent with spike-timing-dependent plasticity (STDP), thus providing a theoretical ground for the systematic design of biologically inspired networks with large and robust long-range sequence storage capacity.
EDIT: Exemplar-Domain Aware Image-to-Image Translation

Image-to-image translation is to convert an image of the certain style to another of the target style with the content preserved. A desired translator should be capable to generate diverse results in a controllable (many-to-many) fashion. To this end, we design a novel generative adversarial network, namely exemplar-domain aware image-to-image translator (EDIT for short). The principle behind is that, for images from multiple domains, the content features can be obtained by a uniform extractor, while (re-)stylization is achieved by mapping the extracted features specifically to different purposes (domains and exemplars). The generator of our EDIT comprises of a part of blocks configured by shared parameters, and the rest by varied parameters exported by an exemplar-domain aware parameter network. In addition, a discriminator is equipped during the training phase to guarantee the output satisfying the distribution of the target domain. Our EDIT can flexibly and effectively work on multiple domains and arbitrary exemplars in a unified neat model. We conduct experiments to show the efficacy of our design, and reveal its advances over other state-of-the-art methods both quantitatively and qualitatively.
Reinventing 2D Convolutions for 3D Medical Images

There has been considerable debate over 2D and 3D representation learning on 3D medical images. 2D approaches could benefit from large-scale 2D pretraining, whereas they are generally weak in capturing large 3D contexts. 3D approaches are natively strong in 3D contexts, however few publicly available 3D medical dataset is large and diverse enough for universal 3D pretraining. Even for hybrid (2D + 3D) approaches, the intrinsic disadvantages within the 2D / 3D parts still exist. In this study, we bridge the gap between 2D and 3D convolutions by reinventing the 2D convolutions. We propose ACS (axial-coronal-sagittal) convolutions to perform natively 3D representation learning, while utilizing the pretrained weights from 2D counterparts. In ACS convolutions, 2D convolution kernels are split by channel into three parts, and convoluted separately on the three views (axial, coronal and sagittal) of 3D representations. Theoretically, ANY 2D CNN (ResNet, DenseNet, or DeepLab) is able to be converted into a 3D ACS CNN, with pretrained weights of same parameter sizes. Extensive experiments on proof-of-concept dataset and several medical benchmarks validate the consistent superiority of the pretrained ACS CNNs, over the 2D / 3D CNN counterparts with / without pretraining. Even without pretraining, the ACS convolution can be used as a plug-and-play replacement of standard 3D convolution, with smaller model size.
Inclusion among commutators of elementary subgroups.

In the present paper we continue the study of the elementary commutator subgroups $[E(n,A),E(n,B)]$, where $A$ and $B$ are two-sided ideals of an associative ring $R$, $n\ge 3$. First, we refine and expand a number of the auxiliary results, both classical ones, due to Bass, Stein, Mason, Stothers, Tits, Vaserstein, van der Kallen, Stepanov, as also some of the intermediate results in our joint works with Hazrat, and our own recent papers [40,41]. The gimmick of the present paper is an explicit triple congruence for elementary commutators $[t_{ij}(ab),t_{ji}(c)]$, where $a,b,c$ belong to three ideals $A,B,C$ of $R$. In particular, it provides a sharper counterpart of the three subgroups lemma at the level of ideals. We derive some further striking corollaries thereof, such as a complete description of generic lattice of commutator subgroups $[E(n,I^r),E(n,I^s)]$, new inclusions among multiple elementary commutator subgroups, etc.
A Benchmark for Anomaly Segmentation

Detecting out-of-distribution examples is important for safety-critical machine learning applications such as self-driving vehicles. However, existing research mainly focuses on small-scale images where the whole image is considered anomalous. We propose to segment only the anomalous regions within an image, and hence we introduce the Combined Anomalous Object Segmentation benchmark for the more realistic task of large-scale anomaly segmentation. Our benchmark combines two novel datasets for anomaly segmentation that incorporate both realism and anomaly diversity. Using both real images and those from a simulated driving environment, we ensure the background context and a wide variety of anomalous objects are naturally integrated, unlike before. Additionally, we improve out-of-distribution detectors on large-scale multi-class datasets and introduce detectors for the previously unexplored setting of multi-label out-of-distribution detection. These novel baselines along with our anomaly segmentation benchmark open the door to further research in large-scale out-of-distribution detection and segmentation.
PlantDoc: A Dataset for Visual Plant Disease Detection

India loses 35% of the annual crop yield due to plant diseases. Early detection of plant diseases remains difficult due to the lack of lab infrastructure and expertise. In this paper, we explore the possibility of computer vision approaches for scalable and early plant disease detection. The lack of availability of sufficiently large-scale non-lab data set remains a major challenge for enabling vision based plant disease detection. Against this background, we present PlantDoc: a dataset for visual plant disease detection. Our dataset contains 2,598 data points in total across 13 plant species and up to 17 classes of diseases, involving approximately 300 human hours of effort in annotating internet scraped images. To show the efficacy of our dataset, we learn 3 models for the task of plant disease classification. Our results show that modelling using our dataset can increase the classification accuracy by up to 31%. We believe that our dataset can help reduce the entry barrier of computer vision techniques in plant disease detection.
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild

We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision. The method is based on an autoencoder that factors each input image into depth, albedo, viewpoint and illumination. In order to disentangle these components without supervision, we use the fact that many object categories have, at least in principle, a symmetric structure. We show that reasoning about illumination allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, we model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. Our experiments show that this method can recover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, we demonstrate superior accuracy compared to another method that uses supervision at the level of 2D image correspondences.
Rigging the Lottery: Making All Tickets Winners

Many applications require sparse neural networks due to space or inference time restrictions. There is a large body of work on training dense networks to yield sparse networks for inference, but this limits the size of the largest trainable sparse model to that of the largest trainable dense model. In this paper we introduce a method to train sparse neural networks with a fixed parameter count and a fixed computational cost throughout training, without sacrificing accuracy relative to existing dense-to-sparse training methods. Our method updates the topology of the sparse network during training by using parameter magnitudes and infrequent gradient calculations. We show that this approach requires fewer floating-point operations (FLOPs) to achieve a given level of accuracy compared to prior techniques. We demonstrate state-of-the-art sparse training results on a variety of networks and datasets, including ResNet-50, MobileNets on Imagenet-2012, and RNNs on WikiText-103. Finally, we provide some insights into why allowing the topology to change during the optimization can overcome local minima encountered when the topology remains static. Code used in our work can be found in this http URL.
Narrow-band Deep Filtering for Multichannel Speech Enhancement

In this paper we address the problem of multichan-nel speech enhancement in the short-time Fourier transform (STFT) domain and in the framework of sequence-to-sequence deep learning. A long short-time memory (LSTM) network takes as input a sequence of STFT coefficients associated with a frequency bin of multichannel noisy-speech signals. The net-work's output is a sequence of single-channel cleaned speech at the same frequency bin. We propose several clean-speech network targets, namely, the magnitude ratio mask, the complex ideal ratio mask, the STFT coefficients and spatial filtering. A prominent feature of the proposed model is that the same LSTM architecture, with identical parameters, is trained across frequency bins. The proposed method is referred to as narrow-band deep filtering. This choice stays in contrast with traditional wide-band speech enhancement methods. The proposed deep filter is able to discriminate between speech and noise by exploiting their different temporal and spatial characteristics: speech is non-stationary and spatially coherent while noise is relatively stationary and weakly correlated across channels. This is similar in spirit with unsupervised techniques, such as spectral subtraction and beamforming. We describe extensive experiments with both mixed signals (noise is added to clean speech) and real signals (live recordings). We empirically evaluate the proposed architecture variants using speech enhancement and speech recognition metrics, and we compare our results with the results obtained with several state of the art methods. In the light of these experiments we conclude that narrow-band deep filtering has very good performance, and excellent generalization capabilities in terms of speaker variability and noise type.
Ferro-advection aided evaporation kinetics of ferrofluid droplets in magnetic field ambience

The present article discusses the physics and mechanics of evaporation of pendent, aqueous ferrofluid droplets and modulation of the same by external magnetic field. We show experimentally and by mathematical analysis that the presence of magnetic field improves the evaporation rates of ferrofluid droplets. First we tackle the question of improved evaporation of the colloidal droplets compared to water, and propose physical mechanisms to explain the same. Experiments show that the changes in evaporation rates aided by the magnetic field cannot be explained on the basis of changes in surface tension, or based on classical diffusion driven evaporation models. Probing using particle image velocimetry shows that the internal advection kinetics of such droplets plays a direct role towards the augmented evaporation rates by modulating the associated Stefan flow. Infrared thermography reveal changes in the thermal gradients within the droplet and evaluating the dynamic surface tension reveals presence of solutal gradients within the droplet, both brought about by the external field. Based on the premise, a scaling analysis of the internal magnetothermal and magnetosolutal ferroadvection behavior is presented. The model incorporates the role of the governing Hartmann number, the magnetothermal Prandtl number and the magnetosolutal Schmidt number. The analysis and stability maps reveal that the magneto-solutal ferroadvection is the more dominant mechanism, and the model is able to predict the internal advection velocities with accuracy. Further, another scaling model to predict the modified Stefan flow is proposed, and is found to accurately predict the improved evaporation rates.
Visualizing point cloud classifiers by curvature smoothing

Recently, several networks that operate directly on point clouds have been proposed. There is significant utility in understanding their mechanisms to classify point clouds, which can potentially help diagnosing these networks and designing better architectures. In this paper, we propose a novel approach to visualize features important to the point cloud classifiers. Our approach is based on smoothing curved areas on a point cloud. After prominent features were smoothed, the resulting point cloud can be evaluated on the network to assess whether the feature is important to the classifier. A technical contribution of the paper is an approximated curvature smoothing algorithm, which can smoothly transition from the original point cloud to one of constant curvature, such as a uniform sphere. Based on the smoothing algorithm, we propose PCI-GOS (Point Cloud Integrated-Gradients Optimized Saliency), a visualization technique that can automatically find the minimal saliency map that covers the most important features on a shape. Experiment results revealed insights into different point cloud classifiers.
Ionized gas outflow signatures in SDSS-IV MaNGA active galactic nuclei

The prevalence of outflow and feedback signatures in active galactic nuclei (AGNs is a major unresolved question which large integral field unit (IFU) surveys now allow to address. In this paper, we present a kinematic analysis of the ionized gas in 2778 galaxies at z ∼ 0.05 observed by Sloan Digital Sky Survey-IV (SDSS-IV) Mapping Nearby Galaxies at Apache Point Observatory (MaNGA). Specifically, we measure the kinematics of the [O iii] λ5007 Å emission line in each spatial element and fit multiple Gaussian components to account for possible non-gravitational motions of gas. Comparing the kinematics of the ionized gas between 308 MaNGA-selected AGNs that have been previously identified through emission-line diagnostics and sources not classified as AGN, we find that while 25 per cent of MaNGA-selected AGN show [O iii] components with emission-line widths of >500 km s−1 in more than 10 per cent of their spaxels, only 7 per cent of MaNGA non-AGNs show a similar signature. Even the AGNs that do not show nuclear AGN photoionization signatures and that were only identified as AGN based on their larger scale photoionization signatures show similar kinematic characteristics. In addition to obscuration, another possibility is that outflow and mechanical feedback signatures are longer lived than the AGN itself. Our measurements demonstrate that high velocity gas is more prevalent in AGN compared to non-AGN and that outflow and feedback signatures in low-luminosity, low-redshift AGN may so far have been underestimated. We show that higher luminosity MaNGA-selected AGNs are able to drive larger scale outflows than lower luminosity AGN. But estimates of the kinetic coupling efficiencies are ≪1 per cent and suggest that the feedback signatures probed in this paper are unlikely to have a significant impact on the AGN host galaxies. However, continuous energy injection may still heat a fraction of the cool gas and delay or suppress star formation in individual galaxies even when the AGN is weak.
Normal Assisted Stereo Depth Estimation

Accurate stereo depth estimation plays a critical role in various 3D tasks in both indoor and outdoor environments. Recently, learning-based multi-view stereo methods have demonstrated competitive performance with limited number of views. However, in challenging scenarios, especially when building cross-view correspondences is hard, these methods still cannot produce satisfying results. In this paper, we study how to enforce the consistency between surface normal and depth at training time to improve the performance. We couple the learning of a multi-view normal estimation module and a multi-view depth estimation module. In addition, we propose a novel consistency loss to train an independent consistency module that refines the depths from depth/normal pairs. We find that the joint learning can improve both the prediction of normal and depth, and the accuracy and smoothness can be further improved by enforcing the consistency. Experiments on MVS, SUN3D, RGBD and Scenes11 demonstrate the effectiveness of our method and state-of-the-art performance.
A translational study of ixabepilone (BMS-247550) in renal cell cancer (RCC): Assessment of its activity and demonstration of target engagement in tumor cells

4541 Background: Agents targeting microtubules (MTs) are very effective anticancer drugs. Ixabepilone is a novel MT-stabilizing agent active in taxane-refractory models. Prior studies have shown that α-tubulin undergoes post-translational modification following assembly into MTs, including removal of the C-terminal tyrosine, exposing the penultimate glutamic acid (glu-tubulin) and acetylation of lysine 40. Methods: 67 patients with RCC were enrolled on study. 61 patients had a prior nephrectomy and 26 had received at least one prior systemic immune therapy. Treatment consisted of 6 mg/m2 ixabepilone on days 1–5 of a 21 day cycle. Response evaluations were performed every 2 cycles using RECIST. Patients with accessible tumor had biopsies prior to treatment and following the fifth dose of ixabepilone. Protein immunoblots were prepared and probed with antibodies to total and modified α-tubulins, and to β-actin. Results: 402 cycles have been administered. Grade 1, 2, or 3 neurotoxicity was recorded in 15, 17,.
Explaining Neural Networks via Perturbing Important Learned Features

Attributing the output of a neural network to the contribution of given input elements is one way of shedding light on the black box nature of neural networks. We propose a novel input feature attribution method that finds an input perturbation that maximally changes the output neuron by exclusively perturbing important hidden neurons (i.e. learned features) on the path to output neuron. Given an input, this is achieved by 1) pruning unimportant neurons, and subsequently 2) finding a local input perturbation that maximizes the output in the pruned network. Since our method considers the importance of hidden neurons (high-level features), it inherently considers interdependencies between multiple input elements, which is vital for input feature attribution. We propose PruneGrad, an efficient gradient-based solution for the pruning and perturbation steps of our method. The efficacy of our method is evaluated by quantitatively benchmarking against other attribution methods using 1) sanity checks, 2) pixel perturbation, and 3) Remove and Retrain (ROAR). Our results show that while most of the existing attribution methods are prone to fail or get mediocre results in at least one benchmark, our proposed method achieves state of the art results in all three benchmarks. The results are further supported by comparative visual evaluation.
Linear mappings preserving the copositive cone

Let $\mathcal{S}_n$ be the set of all $n$-by-$n$ symmetric real matrices, and let $\mathcal{C}_n$ be the copositive cone, that is, the set of all matrices $a\in\mathcal{S}_n$ that fulfill the condition $u^\top a u\geqslant0$ for all $n$-vectors $u$ with nonnegative entries. We prove that a linear mapping $\varphi:\mathcal{S}_n\to \mathcal{S}_n$ satisfies $\varphi(\mathcal{C}_n)=\mathcal{C}_n$ if and only if $$\varphi(x)=m^\top xm$$ for a fixed monomial matrix $m$ with nonnegative entries.
Abnormal Returns in Insider Trading – A study on the efficiency of the Swedish stock market and insider trading legislation

The purpose of this paper is to study the efficiency of the Swedish stock market and its insider trading legislation. This is done by measuring both the occurrence and timing of abnormal returns among corporate insiders. In addition, we also study if there are differences in abnormal returns between insiders in companies of different sizes, between purchaseand sale transactions and between different types of insiders. The results show that all types of insiders, in all company sizes and for both purchaseand sale transactions receive positive abnormal returns in the year following their transactions. The levels of abnormal returns vary but are all significant, both statistically and economically. Furthermore, the results show that there is no difference between purchaseand sale transactions, but that there are some differences between companies of different sizes as well as between different types of insiders. Tutor: Henrik Andersson, Assistant Professor * 22566@student.hhs. se ** 40872@student.hhs. se