Miracle-dz/UniFAR-FTUs huggingface dataset:
Error Analysis on Graph Laplacian Regularized Estimator

We provide a theoretical analysis of the representation learning problem aimed at learning the latent variables (design matrix) $\Theta$ of observations $Y$ with the knowledge of the coefficient matrix $X$. The design matrix is learned under the assumption that the latent variables $\Theta$ are smooth with respect to a (known) topological structure $\mathcal{G}$. To learn such latent variables, we study a graph Laplacian regularized estimator, which is the penalized least squares estimator with penalty term proportional to a Laplacian quadratic form. This type of estimators has recently received considerable attention due to its capability in incorporating underlying topological graph structure of variables into the learning process. While the estimation problem can be solved efficiently by state-of-the-art optimization techniques, its statistical consistency properties have been largely overlooked. In this work, we develop a non-asymptotic bound of estimation error under the classical statistical setting, where sample size is larger than the ambient dimension of the latent variables. This bound illustrates theoretically the impact of the alignment between the data and the graph structure as well as the graph spectrum on the estimation accuracy. It also provides theoretical evidence of the advantage, in terms of convergence rate, of the graph Laplacian regularized estimator over classical ones (that ignore the graph structure) in case of a smoothness prior. Finally, we provide empirical results of the estimation error to corroborate the theoretical analysis.
Deep Learning Methods for Event Verification and Image Repurposing Detection

The authenticity of images posted on social media is an issue of growing concern. Many algorithms have been developed to detect manipulated images, but few have investigated the ability of deep neural network based approaches to verify the authenticity of image labels, such as event names. In this paper, we propose several novel methods to predict if an image was captured at one of several noteworthy events. We use a set of images from several recorded events such as storms, marathons, protests, and other large public gatherings. Two strategies of applying pre-trained Imagenet network for event verification are presented, with two modifications for each strategy. The first method uses the features from the last convolutional layer of a pre-trained network as input to a classifier. We also consider the effects of tuning the convolutional weights of the pre-trained network to improve classification. The second method combines many features extracted from smaller scales and uses the output of a pre-trained network as the input to a second classifier. For both methods, we investigated several different classifiers and tested many different pre-trained networks. Our experiments demonstrate both these approaches are effective for event verification and image re-purposing detection. The classification at the global scale tends to marginally outperform our tested local methods and fine tuning the network further improves the results.
Adversarially Trained Model Compression: When Robustness Meets Efficiency

The robustness of deep models to adversarial attacks has gained significant attention in recent years, so has the model compactness and efficiency: yet the two have been mostly studied separately, with few relationships drawn between each other. This paper is concerned with: how can we combine the best of both worlds, obtaining a robust and compact network? The answer is not as straightforward as it may seem, since the two goals of model robustness and compactness may contradict from time to time. We formally study this new question, by proposing a novel Adversarially Trained Model Compression (ATMC) framework. A unified constrained optimization formulation is designed, with an efficient algorithm developed. An extensive group of experiments are then carefully designed and presented, demonstrating that ATMC obtains remarkably more favorable trade-off among model size, accuracy and robustness, over currently available alternatives in various settings.
Differential Similarity in Higher Dimensional Spaces: Theory and Applications

This paper presents an extension and an elaboration of the theory of differential similarity, which was originally proposed in arXiv:1401.2411 [cs.LG]. The goal is to develop an algorithm for clustering and coding that combines a geometric model with a probabilistic model in a principled way. For simplicity, the geometric model in the earlier paper was restricted to the three-dimensional case. The present paper removes this restriction, and considers the full $n$-dimensional case. Although the mathematical model is the same, the strategies for computing solutions in the $n$-dimensional case are different, and one of the main purposes of this paper is to develop and analyze these strategies. Another main purpose is to devise techniques for estimating the parameters of the model from sample data, again in $n$ dimensions. We evaluate the solution strategies and the estimation techniques by applying them to two familiar real-world examples: the classical MNIST dataset and the CIFAR-10 dataset.
Dropout as a Structured Shrinkage Prior

Dropout regularization of deep neural networks has been a mysterious yet effective tool to prevent overfitting. Explanations for its success range from the prevention of "co-adapted" weights to it being a form of cheap Bayesian inference. We propose a novel framework for understanding multiplicative noise in neural networks, considering continuous distributions as well as Bernoulli noise (i.e. dropout). We show that multiplicative noise induces structured shrinkage priors on a network's weights. We derive the equivalence through reparametrization properties of scale mixtures and without invoking any approximations. Given the equivalence, we then show that dropout's Monte Carlo training objective approximates marginal MAP estimation. We leverage these insights to propose a novel shrinkage framework for resnets, terming the prior 'automatic depth determination' as it is the natural analog of automatic relevance determination for network depth. Lastly, we investigate two inference strategies that improve upon the aforementioned MAP approximation in regression benchmarks.
Multi-Dimensional Balanced Graph Partitioning via Projected Gradient Descent

Motivated by performance optimization of large-scale graph processing systems that distribute the graph across multiple machines, we consider the balanced graph partitioning problem. Compared to most of the previous work, we study the multi-dimensional variant in which balance according to multiple weight functions is required. As we demonstrate by experimental evaluation, such multi-dimensional balance is essential for achieving performance improvements for typical distributed graph processing workloads. We propose a new scalable technique for the multidimensional balanced graph partitioning problem. It is based on applying randomized projected gradient descent to a non-convex continuous relaxation of the objective. We show how to implement the new algorithm efficiently in both theory and practice utilizing various approaches for the projection step. Experiments with large-scale graphs containing up to hundreds of billions of edges indicate that our algorithm has superior performance compared to the state of the art.
Unpaired Image-to-Image Translation with Domain Supervision

Image-to-image translation has been widely investigated in recent years. Existing approaches are elaborately designed in an unsupervised manner and little attention has been paid to domain information beneath unpaired data. In this work, we treat domain information as explicit supervision and design an unpaired image-to-image translation framework, Domain-supervised GAN (briefly, DosGAN), that takes the first step towards exploration of domain supervision. Instead of representing domain characteristics with different generators in CycleGAN [32] or multiple domain codes in StarGAN [2], we pre-train a classification network to classify the domain of an image. After pre-training, this network is used to extract domain features of each image by using the output of its second-to-last layer. Such features, together with the latent semantic features extracted by another encoder (shared across different domains), are used to generate an image in the target domain. Experiments on multiple hair color translation, multiple identity translation and conditional edges-to-shoes/handbags demonstrate the effectiveness of our method. In addition, we transfer the domain feature extractor obtained on the Facescrub dataset with domain supervision information, to the CelebA dataset without domain supervision information, and succeed achieving conditional translation with any two images in CelebA, while previous models like StarGAN cannot handle this task. Our code is available at https://github.com/ linjx-ustc1106/DosGAN-PyTorch.
A Game of Drones: Cyber-Physical Security of Time-Critical UAV Applications With Cumulative Prospect Theory Perceptions and Valuations

In this paper, a novel mathematical framework is introduced for modeling and analyzing the cyber-physical security of time-critical UAV applications. A general UAV security network interdiction game is formulated to model interactions between a UAV operator and an interdictor, each of which can be benign or malicious. In this game, the interdictor chooses the optimal location(s) from which to target the drone system by interdicting the potential paths of the UAVs. Meanwhile, the UAV operator responds by finding an optimal path selection policy that enables its UAVs to evade attacks and minimize their mission completion time. New notions from cumulative prospect theory (PT) are incorporated into the game to capture the operator’s and the interdictor’s subjective valuations of mission completion times and perceptions of the risk levels facing the UAVs. The equilibrium of the game, with and without PT, is then analytically characterized and studied, while providing detailed derivations of mission completion times (and their expected values), PT valuations of both players (along with proofs of their convergence), and equilibrium points under different studied security regimes. Novel algorithms are then proposed to reach the game’s equilibria under both PT and classical game theory. Simulation results show the properties of the equilibrium for both the rational and PT cases, highlighting the effects of bounded rationality on the interdictor’s and the operator’s strategies as well as mission completion times, and the way it can be exploited by a fully rational opponent.
Improved Knowledge Distillation via Teacher Assistant: Bridging the Gap Between Student and Teacher

Despite the fact that deep neural networks are powerful models and achieve appealing results on many tasks, they are too gigantic to be deployed on edge devices like smart-phones or embedded sensor nodes. There has been efforts to compress these networks, and a popular method is knowledge distillation, where a large ( a.k.a. teacher) pre-trained network is used to train a smaller ( a.k.a. student) network. However, in this paper, we show that the student network performance de-grades when the gap between student and teacher is large. Given a ﬁxed student network, one cannot employ an arbitrarily large teacher, or in other words, a teacher can effectively transfer its knowledge to students up to a certain size, not smaller. To alleviate this shortcoming, we introduce multi-step knowledge distillation which employs an intermediate-sized network ( a.k.a. teacher assistant) to bridge the gap between the student and the teacher. We study the effect of teacher assistant size and extend the framework to multi-step distillation. Moreover, empirical and theoretical analysis are conducted to analyze the teacher assistant knowledge distillation framework. Extensive experiments on CIFAR-10 and CIFAR-100 datasets and plain CNN and ResNet architectures substantiate the effectiveness of our proposed approach.
Doing things with being undone

Mis-hear the ‘cene’ in Anthropocene and we are not beholders of an epoch or witnesses to a prospect of distancing projection onto a deep past or lost future but, rather, in the scene of our undoing. In this scene that I reframe as the Necrocene, there are still ways of doing things with being undone. Current art practice offers a new ars moriendi to make contestatorily palpable and even transform the necropolitical conditions of the Necrocene crisis by working with the strangely resilient powers of death. Current practices that deform the landscape-form demonstrate how the vulnerability of living our dying offers a queer material medium to agitate for livable life toward a black, trans* more-than-human commons.
Exchangeable and sampling-consistent distributions on rooted binary trees

Abstract We introduce a notion of finite sampling consistency for phylogenetic trees and show that the set of finitely sampling-consistent and exchangeable distributions on n-leaf phylogenetic trees is a polytope. We use this polytope to show that the set of all exchangeable and sampling-consistent distributions on four-leaf phylogenetic trees is exactly Aldous’ beta-splitting model, and give a description of some of the vertices for the polytope of distributions on five leaves. We also introduce a new semialgebraic set of exchangeable and sampling consistent models we call the multinomial model and use it to characterize the set of exchangeable and sampling-consistent distributions. Using this new model, we obtain a finite de Finetti-type theorem for rooted binary trees in the style of Diaconis’ theorem on finite exchangeable sequences.
Photorealistic Image Synthesis for Object Instance Detection

We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images. The proposed approach has three key ingredients: (1) 3D object models are rendered in 3D models of complete scenes with realistic materials and lighting, (2) plausible geometric configuration of objects and cameras in a scene is generated using physics simulation, and (3) high photorealism of the synthesized images is achieved by physically based rendering. When trained on images synthesized by the proposed approach, the Faster R-CNN object detector [1] achieves a 24% absolute improvement of mAP@. 75IoU on Rutgers APC [2] and 11% on LineMod-Occluded [3] datasets, compared to a baseline where the training images are synthesized by rendering object models on top of random photographs. This work is a step towards being able to effectively train object detectors without capturing or annotating any real images. A dataset of 400K synthetic images with ground truth annotations for various computer vision tasks will be released on the project website: thodan.github. io/objectsynth.
Cell-Free Massive MIMO for UAV Communications

We study support for unmanned aerial vehicle (UAV) communications through a cell-free massive MIMO architecture. Under the general assumption that the propagation channel between the mobile stations, either UAVs or ground users, and the access points follows a Ricean distribution, we derive closed form spectral efficiency lower bounds for uplink and downlink with linear minimum mean square error (LMMSE) channel estimation. We also propose power allocation and user scheduling strategies for such a system. Our numerical results reveal that a cell-free massive MIMO architecture may provide better performance than a traditional multicell massive MIMO network deployment.
Autonomous Conflict Resolution Method for multi-UAVs Based on Preorder Flight Information

In complex low altitude airspace, some factors may cause that flight information cannot interact properly between UAVs. UAV cannot normally obtain flight information of others within its communication range. Therefore, we propose a distributed approach to multi-aircraft real-time conflict resolution. UAVs can enhance the ability of autonomous conflict resolution through the study of preorder flight information with the prerequisite for the unavailability of others' flight information. First, we obtain the payoff of safety and efficiency adopting speed and heading adjustments as maneuver strategy of collision avoidance. Then, according to the preorder flight information, UAV builds a memory pool which is learned by it to obtain the most rewarding collision avoidance strategy. Finally, the simulation results on the random flight scenario of two concentric circles show that our approach is promising in terms of safety and system efficiency.
FrAppLe: A Framework for Apprenticeship Learning

Human and AI collaboration has evolved drastically over the past decade. Most of the tasks are now partly handled by machines and partly by humans, resulting in human and AI partnership. Humans share their tasks with the AI agents as they are good at performing predictive and repetitive tasks. The tasks assigned to the agents can be dynamic in nature and can change over time. This may lead to acquiring new skills or re-training to improvise on the same task. The developer needs to programme these agents every time for a new domain. In this work, we propose a framework that helps in developing apprentice agents that learn from Human demonstration. Apprenticeship learning is the process of learning by observing an expert performing actions to achieve a specific goal. The agent observes the expert's actions, the sensory inputs and then trains itself using the observed data. We also propose a process to learn and self-diagnose the agent's action for apprenticeship learning. The framework alleviates the need to explicitly program the apprentice agents. We demonstrate our framework on a case study.
ReStoCNet: Residual Stochastic Binary Convolutional Spiking Neural Network for Memory-Efficient Neuromorphic Computing

In this work, we propose ReStoCNet, a residual stochastic multilayer convolutional Spiking Neural Network (SNN) composed of binary kernels, to reduce the synaptic memory footprint and enhance the computational efficiency of SNNs for complex pattern recognition tasks. ReStoCNet consists of an input layer followed by stacked convolutional layers for hierarchical input feature extraction, pooling layers for dimensionality reduction, and fully-connected layer for inference. In addition, we introduce residual connections between the stacked convolutional layers to improve the hierarchical feature learning capability of deep SNNs. We propose Spike Timing Dependent Plasticity (STDP) based probabilistic learning algorithm, referred to as Hybrid-STDP (HB-STDP), incorporating Hebbian and anti-Hebbian learning mechanisms, to train the binary kernels forming ReStoCNet in a layer-wise unsupervised manner. We demonstrate the efficacy of ReStoCNet and the presented HB-STDP based unsupervised training methodology on the MNIST and CIFAR-10 datasets. We show that residual connections enable the deeper convolutional layers to self-learn useful high-level input features and mitigate the accuracy loss observed in deep SNNs devoid of residual connections. The proposed ReStoCNet offers >20 × kernel memory compression compared to full-precision (32-bit) SNN while yielding high enough classification accuracy on the chosen pattern recognition tasks.
Lyapunov Criterion for Stochastic Systems and Its Applications in Distributed Computation

This paper presents new sufficient conditions for convergence and asymptotic or exponential stability of a stochastic discrete-time system, under which the constructed Lyapunov function always decreases in expectation along the system's solutions after a finite number of steps, but without necessarily strict decrease at every step, in contrast to the classical stochastic Lyapunov theory. As the first application of this new Lyapunov criterion, we look at the product of any random sequence of stochastic matrices, including those with zero diagonal entries, and obtain sufficient conditions to ensure the product almost surely converges to a matrix with identical rows; we also show that the rate of convergence can be exponential under additional conditions. As the second application, we study a distributed network algorithm for solving linear algebraic equations. We relax existing conditions on the network structures, while still guaranteeing the equations are solved asymptotically.
Could a tangible interface help a child to weigh his/her opinion on usability?

Research in the Human-Computer Interaction field needs to design interactions adapted for users and to rate opinion or personal perception of the interactions. Applied to the education field, one of the challenges of HCI is that it relies on the design of user-interfaces and interactions designed to meet pupils and teachers' requirements. To evaluate designed interactions, researchers currently use scale-based questionnaires like Likert's. For now, most of the time data collection is different between pupils and teachers. Indeed, young respondents have less experience than adults and could be more limited in their ability to express themselves. Several studies show that children tend to select extreme answers on these scales. To reduce this bias, we investigated the role of tangibility. We developed a prototypical device to help them to "weigh" their opinion. We conducted a preliminary study with children. First, our results show that our tangible interface seems to engage young respondents to metaanalyze system usability. Secondly our prototype seems to reduce extreme responses.
Thyroid Hormones and Cancer: A Comprehensive Review of Preclinical and Clinical Studies

Thyroid hormones take major part in normal growth, development and metabolism. Over a century of research has supported a relationship between thyroid hormones and the pathophysiology of various cancer types. In vitro studies as well as research in animal models demonstrated an effect of the thyroid hormones T3 and T4 on cancer proliferation, apoptosis, invasiveness and angiogenesis. Thyroid hormones mediate their effects on the cancer cell through several non-genomic pathways including activation of the plasma membrane receptor integrin αvβ3. Furthermore, cancer development and progression are affected by dysregulation of local bioavailability of thyroid hormones. Case-control and population-based studies provide conflicting results regarding the association between thyroid hormones and cancer. However, a large body of evidence suggests that subclinical and clinical hyperthyroidism increase the risk of several solid malignancies while hypothyroidism may reduce aggressiveness or delay the onset of cancer. Additional support is provided from studies in which dysregulation of the thyroid hormone axis secondary to cancer treatment or thyroid hormone supplementation was shown to affect cancer outcomes. Recent preclinical and clinical studies in various cancer types have further shown promising outcomes following chemical reduction of thyroid hormones or inhibition or their binding to the integrin receptor. This review provides a comprehensive overview of the preclinical and clinical research conducted so far.