Miracle-dz/UniFAR-FTUs huggingface dataset:
Low-Budget Unsupervised Label Query through Domain Alignment Enforcement

Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities. Despite the public availability of a large quantity of datasets, it is often necessary to generate a new set of labelled data to address specific requirements. In addition, the production of labels is costly and sometimes it requires a specific expertise to be fulfilled. In this work, we introduce a new problem called low budget unsupervised label query that consists in a model trained to suggests to the user a set of samples to be labelled, from a completely unlabelled dataset, to maximize the classification accuracy on that dataset. We propose to adopt a domain alignment model, modified to enforce consistency, to align a known dataset (source) and the dataset to be labelled (target). Finally, we propose a novel sample selection method based on uniform entropy sampling, named UNFOLD, which is deterministic and steadily outperforms other baselines as well as competing models on a large variety of publicly available datasets.
Superfluidity and pairing phenomena in ultracold atomic Fermi gases in one-dimensional optical lattices. I. Balanced case

The superfluidity and pairing phenomena in ultracold atomic Fermi gases have been of great interest in recent years, with multiple tunable parameters. Here we study the BCS-BEC crossover behavior of balanced two-component Fermi gases in a one-dimensional optical lattice, which is distinct from the simple three-dimensional (3D) continuum and a fully 3D lattice often found in a condensed matter system. We use a pairing fluctuation theory which includes self-consistent feedback effects at finite temperatures, and find widespread pseudogap phenomena beyond the BCS regime. As a consequence of the lattice periodicity, the superfluid transition temperature $T_c$ decreases with pairing strength in the BEC regime, where it approaches asymptotically $T_c = \pi an/2m$, with $a$ being the $s$-wave scattering length, and $n$ ($m$) the fermion density (mass). In addition, the quasi-two dimensionality leads to fast growing (absolute value of the) fermionic chemical potential $\mu$ and pairing gap $\Delta$, which depends exponentially on the ratio $d/a$. Importantly, $T_c$ at unitarity increases with the lattice constant $d$ and hopping integral $t$. The effect of the van Hove singularity on $T_c$ is identified. The superfluid density exhibits $T^{3/2}$ power laws at low $T$, away from the extreme BCS limit. These predictions can be tested in future experiments.
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks

The rapid development of deep neural networks (DNNs) in recent years can be attributed to the various techniques that address gradient explosion and vanishing. In order to understand the principle behind these techniques and develop new methods, plenty of metrics have been proposed to identify networks that are free of gradient explosion and vanishing. However, due to the diversity of network components and complex serial-parallel hybrid connections in modern DNNs, the evaluation of existing metrics usually requires strong assumptions, complex statistical analysis, or has limited application fields, which constraints their spread in the community. In this paper, inspired by the Gradient Norm Equality and dynamical isometry, we first propose a novel metric called Block Dynamical Isometry, which measures the change of gradient norm in individual blocks. Because our Block Dynamical Isometry is norm-based, its evaluation needs weaker assumptions compared with the original dynamical isometry. To mitigate challenging derivation, we propose a highly modularized statistical framework based on free probability. Our framework includes several key theorems to handle complex serial-parallel hybrid connections and a library to cover the diversity of network components. Besides, several sufficient conditions for prerequisites are provided. Powered by our metric and framework, we analyze extensive initialization, normalization, and network structures. We find that our Block Dynamical Isometry is a universal philosophy behind them. Then, we improve some existing methods based on our analysis, including an activation function selection strategy for initialization techniques, a new configuration for weight normalization, a depth-aware way to derive coefficients in SeLU, and initialization/weight normalization in DenseNet. Moreover, we propose a novel normalization technique named second moment normalization, which has 30 percent fewer computation overhead than batch normalization without accuracy loss and has better performance under micro batch size. Last but not least, our conclusions and methods are evidenced by extensive experiments on multiple models over CIFAR-10 and ImageNet.
Blockchain for IOT-based NANs and HANs in Smart Grid

Smart Grid, an intelligent connected grid consisting of millions of smart devices, used to collect data from the grid to improve the efficiency of its operation. These smart devices communicating wirelessly are susceptible to attacks and hence the security and privacy of the smart devices along with the smart grid is a major challenge. Blockchain-based systems provide improved security and privacy and hence gained a lot of attention in the recent past. This paper proposes a blockchain-based architecture for Neighborhood Area Networks (NANs) and Home Area networks (HANs) in Smart Grid. The paper presents a security analysis in terms of confidentiality, integrity and availability to show that the proposed blockchain-based Smart Grid architecture is secure. Also, the impact of the improved security on packet delays, energy and computational overhead is discussed.
Options of Interest: Temporal Abstraction with Interest Functions

Temporal abstraction refers to the ability of an agent to use behaviours of controllers which act for a limited, variable amount of time. The options framework describes such behaviours as consisting of a subset of states in which they can initiate, an internal policy and a stochastic termination condition. However, much of the subsequent work on option discovery has ignored the initiation set, because of difficulty in learning it from data. We provide a generalization of initiation sets suitable for general function approximation, by defining an interest function associated with an option. We derive a gradient-based learning algorithm for interest functions, leading to a new interest-option-critic architecture. We investigate how interest functions can be leveraged to learn interpretable and reusable temporal abstractions. We demonstrate the efficacy of the proposed approach through quantitative and qualitative results, in both discrete and continuous environments.
No Spurious Local Minima in Deep Quadratic Networks

Despite their practical success, a theoretical understanding of the loss landscape of neural networks has proven challenging due to the high-dimensional, non-convex, and highly nonlinear structure of such models. In this paper, we characterize the training landscape of the quadratic loss landscape for neural networks with quadratic activation functions. We prove existence of spurious local minima and saddle points which can be escaped easily with probability one when the number of neurons is greater than or equal to the input dimension and the norm of the training samples is used as a regressor. We prove that deep overparameterized neural networks with quadratic activations benefit from similar nice landscape properties. Our theoretical results are independent of data distribution and fill the existing gap in theory for two-layer quadratic neural networks. Finally, we empirically demonstrate convergence to a global minimum for these problems.
"Happy and Assured that life will be easy 10years from now.": Perceptions of Artificial Intelligence in 8 Countries

As the influence and use of artificial intelligence (AI) have grown and its transformative potential has become more apparent, many questions have been raised regarding the economic, political, and social implications of its use. Public opinion plays an important role in these discussions, influencing product adoption, commercial development, research funding, and regulation. In this paper we present results of a survey of public opinion of artificial intelligence conducted with 10,005 respondents spanning eight countries and six continents. We report widespread perception that AI will have significant impact on society, and highlight public optimism about applications for healthcare as well as public concerns about privacy and job loss. We also identify four key sentiments (exciting, useful, worrying, and futuristic) whose prevalence distinguishes response to AI in different countries.
The effect of Bifidobacterium on intestinal bacterial flora and putrefactive products in hemodialysis

透析患者は, その病態ゆえに, 腸内細菌叢の乱れに伴って, 腐敗産物の異常産生が予測される. 今回, 透析患者の腸内細菌叢の検索と, 糞便中の腐敗産物の測定を行ったところ, 好気性菌の占める割合が高くアンモニアなど腐敗産物が高かった. 透析患者に対し腸溶性マイクロカプセルビフィズス菌製剤を投与することにより, 腸内細菌叢の改善と腐敗産物の産生が抑制されたことが判明した. 本研究は透析患者に対して腸溶性マイクロカプセルピフィズス菌製剤の投与を行うことが腸内環境を改善し, 健康維持に役立つ手段の一つと考える.
Stieltjes moment sequences for pattern-avoiding permutations

A small set of combinatorial sequences have coefficients that can be represented as moments of a nonnegative measure on $[0, \infty)$. Such sequences are known as Stieltjes moment sequences. They have a number of nice properties, such as log-convexity, which are useful to rigorously bound their growth constant from below. This article focuses on some classical sequences in enumerative combinatorics, denoted $Av(\mathcal{P})$, and counting permutations of $\{1, 2, \ldots, n \}$ that avoid some given pattern $\mathcal{P}$. For increasing patterns $\mathcal{P}=(12\ldots k)$, we recall that the corresponding sequences, $Av(123\ldots k)$, are Stieltjes moment sequences, and we explicitly find the underlying density function, either exactly or numerically, by using the Stieltjes inversion formula as a fundamental tool. We first illustrate our approach on two basic examples, $Av(123)$ and $Av(1342)$, whose generating functions are algebraic. We next investigate the general (transcendental) case of $Av(123\ldots k)$, which counts permutations whose longest increasing subsequences have length at most $k-1$. We show that the generating functions of the sequences $\, Av(1234)$ and $\, Av(12345)$ correspond, up to simple rational functions, to an order-one linear differential operator acting on a classical modular form given as a pullback of a Gaussian $\, _2F_1$ hypergeometric function, respectively to an order-two linear differential operator acting on the square of a classical modular form given as a pullback of a $\, _2F_1$ hypergeometric function. We demonstrate that the density function for the Stieltjes moment sequence $Av(123\ldots k)$ is closely, but non-trivially, related to the density attached to the distance traveled by a walk in the plane with $k-1$ unit steps in random directions. Finally, we study the challenging case of the $Av(1324)$ sequence and give compelling numerical evidence that this too is a Stieltjes moment sequence. Accepting this, we show how rigorous lower bounds on the growth constant of this sequence can be constructed, which are stronger than existing bounds. A further unproven assumption leads to even better bounds, which can be extrapolated to give an estimate of the (unknown) growth constant.
Nonequilibrium Landau-Zener tunneling in exciton-polariton condensates

For describing a coherent quantum two-level system driven by a linearly time-dependent energy separation of the diabatic states, the Landau-Zener model is routine to serve as a textbook model for its dynamics. Along this research line, a particularly intriguing question is whether the framework of Landau-Zener theory can be extended to an intrinsically nonequilibrium quantum system with coherent and dissipative dynamics occurring on an equal footing. In this work, we are motivated to investigate the Landau-Zener problem of polariton condensates in a periodic potential under nonresonant pumping by using the driven-dissipative Gross-Pitaevskii equations coupled to the rate equation. Within the two-mode approximation, a nonequilibrium Landau-Zener model, characterized by coherent and dissipative dynamics occurring on an equal footing, is derived. Fundamentally different from the previous Landau-Zener model, the total density of nonequilibrium Landau-Zener model, in general, is not the conserved quantity anymore due to the dissipative nature. In a surprise, the parameter regimes of the total density still being conserved can still be found. The motion of Hamiltonian of the nonequilibrium Landau-Zener problem in phase space is further discussed. The instability of the band structure can also be studied by the curvatures in phase space, and there may be two loops in the middle of the Brillouin zone. Detailed analysis of the nonequilibrium nature on the tunneling rate will open a new perspective toward understanding the Landau-Zener problem.
Ensembles of Many Diverse Weak Defenses can be Strong: Defending Deep Neural Networks Against Adversarial Attacks

Despite achieving state-of-the-art performance across many domains, machine learning systems are highly vulnerable to subtle adversarial perturbations. Although defense approaches have been proposed in recent years, many have been bypassed by even weak adversarial attacks. An early study~\cite{he2017adversarial} shows that ensembles created by combining multiple weak defenses (i.e., input data transformations) are still weak. We show that it is indeed possible to construct effective ensembles using weak defenses to block adversarial attacks. However, to do so requires a diverse set of such weak defenses. In this work, we propose Athena, an extensible framework for building effective defenses to adversarial attacks against machine learning systems. Here we conducted a comprehensive empirical study to evaluate several realizations of Athena. More specifically, we evaluated the effectiveness of 5 ensemble strategies with a diverse set of many weak defenses that comprise transforming the inputs (e.g., rotation, shifting, noising, denoising, and many more) before feeding them to target deep neural network (DNN) classifiers. We evaluate the effectiveness of the ensembles with adversarial examples generated by 9 various adversaries (i.e., FGSM, CW, etc.) in 4 threat models (i.e., zero-knowledge, black-box, gray-box, white-box) on MNIST. We also explain, via a comprehensive empirical study, why building defenses based on the idea of many diverse weak defenses works, when it is most effective, and what its inherent limitations and overhead are.
Rethinking Defeasible Reasoning: A Scalable Approach

Abstract Recent technological advances have led to unprecedented amounts of generated data that originate from the Web, sensor networks, and social media. Analytics in terms of defeasible reasoning – for example, for decision making – could provide richer knowledge of the underlying domain. Traditionally, defeasible reasoning has focused on complex knowledge structures over small to medium amounts of data, but recent research efforts have attempted to parallelize the reasoning process over theories with large numbers of facts. Such work has shown that traditional defeasible logics come with overheads that limit scalability. In this work, we design a new logic for defeasible reasoning, thus ensuring scalability by design. We establish several properties of the logic, including its relation to existing defeasible logics. Our experimental results indicate that our approach is indeed scalable and defeasible reasoning can be applied to billions of facts.
Practical Issues of Energy Harvesting and Data Transmissions in Sustainable IoT

The sustainable Internet of Things (IoT) is becoming a promising solution for the green living and smart industries. In this article, we investigate the practical issues in the radio energy harvesting and data communication systems through extensive field experiments. A number of important characteristics of energy harvesting circuits and communication modules have been studied, including the non-linear energy consumption of the communication system relative to the transmission power, the wake-up time associated with the payload, and the varying system power during consecutive packet transmissions. In order to improve the efficiency of energy harvest and energy utilization, we propose a new model to accurately describe the energy harvesting process and the power consumption for sustainable IoT devices. Experiments are performed using commercial IoT devices and RF energy harvesters to verify the accuracy of the proposed model. The experiment results show that the new model matches the performance of sustainable IoT devices very well in the real scenario.
Pre-trained Contextual Embedding of Source Code

The source code of a program not only serves as a formal description of an executable task, but it also serves to communicate developer intent in a human-readable form. To facilitate this, developers use meaningful identifier names and natural-language documentation. This makes it possible to successfully apply sequence-modeling approaches, shown to be effective in natural-language processing, to source code. A major advancement in natural-language understanding has been the use of pre-trained token embeddings; BERT and other works have further shown that pre-trained contextual embeddings can be extremely powerful and can be fine-tuned effectively for a variety of downstream supervised tasks. Inspired by these developments, we present the first attempt to replicate this success on source code. We curate a massive corpus of Python programs from GitHub to pre-train a BERT model, which we call Code Understanding BERT (CuBERT). We also pre-train Word2Vec embeddings on the same dataset. We create a benchmark of five classification tasks and compare fine-tuned CuBERT against sequence models trained with and without the Word2Vec embeddings. Our results show that CuBERT outperforms the baseline methods by a margin of 2.9-22%. We also show its superiority when fine-tuned with smaller datasets, and over fewer epochs. We further evaluate CuBERT's effectiveness on a joint classification, localization and repair task involving prediction of two pointers.
Uncertainty-Based Out-of-Distribution Classification in Deep Reinforcement Learning

Robustness to out-of-distribution (OOD) data is an important goal in building reliable machine learning systems. Especially in autonomous systems, wrong predictions for OOD inputs can cause safety critical situations. As a first step towards a solution, we consider the problem of detecting such data in a value-based deep reinforcement learning (RL) setting. Modelling this problem as a one-class classification problem, we propose a framework for uncertainty-based OOD classification: UBOOD. It is based on the effect that an agent's epistemic uncertainty is reduced for situations encountered during training (in-distribution), and thus lower than for unencountered (OOD) situations. Being agnostic towards the approach used for estimating epistemic uncertainty, combinations with different uncertainty estimation methods, e.g. approximate Bayesian inference methods or ensembling techniques are possible. We further present a first viable solution for calculating a dynamic classification threshold, based on the uncertainty distribution of the training data. Evaluation shows that the framework produces reliable classification results when combined with ensemble-based estimators, while the combination with concrete dropout-based estimators fails to reliably detect OOD situations. In summary, UBOOD presents a viable approach for OOD classification in deep RL settings by leveraging the epistemic uncertainty of the agent's value function.
Quantum walks: The mean first detected transition time

We consider the quantum first detection problem for a particle evolving on a graph under repeated projective measurements with fixed rate $1/\tau$. A general formula for the mean first detected transition time is obtained for a quantum walk in a finite-dimensional Hilbert space where the initial state $|\psi_{\rm in}\rangle$ of the walker is orthogonal to the detected state $|\psi_{\rm d}\rangle$. We focus on diverging mean transition times, where the total detection probability exhibits a discontinuous drop of its value, by mapping the problem onto a theory of fields of classical charges located on the unit disk. Close to the critical parameter of the model, which exhibits a blow-up of the mean transition time, we get simple expressions for the mean transition time. Using previous results on the fluctuations of the return time, corresponding to $|\psi_{\rm in}\rangle = |\psi_{\rm d}\rangle$, we find close to these critical parameters that the mean transition time is proportional to the fluctuations of the return time, an expression reminiscent of the Einstein relation.
Physically Plausible Spectral Reconstruction †

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images, and this spectral reconstruction problem (SR) can often be solved with good (low) error. However, little attention has been paid on whether these models’ behavior can adhere to physics. We show that the leading CNN method introduces unexpected ‘colorimetric errors’, which means the recovered spectra do not reproduce ground-truth RGBs, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most CNN models over-fit to fixed exposure and we demonstrate that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that the physical plausibility of SR is enforced. Remarkably, our physically plausible CNN solutions advance both spectral and colorimetric performance of the original network, while the application of data augmentation trades off the network performance for model stability against varying exposure.
Selection for conditionability in Drosophila melanogaster

Successful bidirectional selective breeding of Drosophila melanogaster for excitatory conditionability is reported, using the reliable measures of individual differences (first described by Holliday & Hirsch, 1984, 1986a, 1986b) to test 1,324 animals. Bidirectional selective breeding for good and poor conditioning has produced, respectively, one population in which the percentage of animals showing good conditioning has increased over 25 generations from 19% to 77% and another in which it has decreased over 23 generations to 0%-4%. No increase in a measure of sensitization induced by an unconditioned stimulus (the central excitatory state) accompanied the increase in the percentage of good conditioners in the population selected for good conditioning, whose level of conditioned responding exceeds that of the sensitization measure.