Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving the Interpretability of Deep Neural Networks with Knowledge Distillation

Deep Neural Networks have achieved huge success at a wide spectrum of applications from language modeling, computer vision to speech recognition. However, nowadays, good performance alone is not enough to satisfy the needs of practical deployment where interpretability is demanded for cases involving ethics and mission critical applications. The complex models of Deep Neural Networks make it hard to understand and reason the predictions, which hinders its further progress. To tackle this problem, we apply the Knowledge Distillation technique to distill Deep Neural Networks into decision trees in order to attain good performance and interpretability simultaneously. We formulate the problem at hand as a multi-output regression problem and the experiments demonstrate that the student model achieves significantly better accuracy performance (about 1% to 5%) than vanilla decision trees at the same level of tree depth. The experiments are implemented on the TensorFlow platform to make it scalable to big datasets. To the best of our knowledge, we are the first to distill Deep Neural Networks into vanilla decision trees on multi-class datasets.
Neural Architecture Search Over a Graph Search Space

Neural Architecture Search (NAS) enabled the discovery of state-of-the-art architectures in many domains. However, the success of NAS depends on the definition of the search space. Current search spaces are defined as a static sequence of decisions and a set of available actions for each decision. Each possible sequence of actions defines an architecture. We propose a more expressive class of search space: directed graphs. In our formalism, each decision is a vertex and each action is an edge. This allows us to model iterative and branching architecture design decisions. We demonstrate in simulation, and on image classification experiments, basic iterative and branching search structures, and show that the graph representation improves sample efficiency.
Finding the proper node ranking method for complex networks

Ranking node importance is crucial in understanding network structure and function on complex networks. Degree, h-index and coreness are widely used, but which one is more proper to a network associated with a dynamical process, e.g. SIR spreading process, is still unclear. To fill this gap, a, which is extracted from the fitting function (f(x)=1-1/(e^(2a(x-b))+1)) of the average number of nodes in each radius of the neighborhood of a node, is proposed. Experiment results which are carried out on twenty real-world networks show that a can classify which of the three measures (degree, h-index and coreness) is more proper to a network in ranking node importance. We also find that [b/3] is a good indicator for forecasting the optimal radius of the neighborhood of a node in ranking node importance for a given network. To the best of our knowledge, it is the first solution of this interesting and open issue. Furthermore, by extending the range of neighborhood where we construct an operator H on of a node, we propose a new method to quantify the importance of a node. The ranking accuracies of most networks can be improved when the radius is increased from 0 to its forecasting optimal radius and the improvement, for the best case, reaches up to 111%. The performances will reduce on half of the networks studied in this paper if we roughly extend the radius of the neighborhood. Our work deepens the understanding of how to find out the proper node ranking method for complex networks. The proposed methods bridge the gaps among network structure and node importance, and may have potential applications in controlling the outbreak of disease, designing of optimal information spreading strategies.
Toward a Self-Learned Smart Contracts

In recent years, Blockchain technology has been highly valued and disruptive. Several researches have presented a merge between blockchain and current application i.e. medical, supply chain, and e-commerce. Although Blockchain architecture does not have a standard yet, IBM, MS, AWS offer BaaS (Blockchain as a Service). In addition to the current public chains i.e. Ethereum, NEO, and Cardeno; there are some differences between several public ledgers in terms of development and architecture. This paper introduces the main factors that affect integration of Artificial Intelligence with Blockchain. As well as, how it could be integrated for forecasting and automating; building self-regulated chain.
Finite State Machines for Semantic Scene Parsing and Segmentation

We introduce in this work a novel stochastic inference process, for scene annotation and object class segmentation, based on finite state machines (FSMs). The design principle of our framework is generative and based on building, for a given scene, finite state machines that encode annotation lattices, and inference consists in finding and scoring the best configurations in these lattices. Different novel operations are defined using our FSM framework including reordering, segmentation, visual transduction, and label dependency modeling. All these operations are combined together in order to achieve annotation as well as object class segmentation.
Efficient Sensitivity Based Reconstruction Technique to Accomplish Breast Hyperelastic Elastography

Hyperelastic models have been acknowledged as constitutive equations which reliably model the nonlinear behaviors observed from soft tissues under various loading conditions. Among them, the Mooney-Rivlin, Yeoh, and polynomial models have been proved capable of accurately modeling responses of breast tissues to applied compressions. Hyperelastic elastography technique takes advantage of the disparities between hyperelastic parameters of varied tissues and the change in hyperelastic parameters in pathological processes. The precise reconstruction of hyperelastic parameters of a completely unknown pathology in the breast in a noninvasive and nondestructive way using the ultrasound elastography has been scrutinized in this paper. In the ultrasound elastography, tissue displacement field is extracted from radio frequency signals or images recorded using the ultrasound medical imaging system; hence the exact displacement field might not be obtained. Our results indicate that the parameters estimated by manipulating the iterative sensitivity-matrix based method converge to tissue's real hyperelastic parameters providing appropriate parameters are assigned to the hypothetical hyperelastic and regularization parameters. Iterative methods have therefore been proposed to compute proper hypothetical hyperelastic and regularization parameters. Accurate estimates of hyperelastic parameters of obscure breast pathology have been achieved even from imprecise measurements of displacements induced in the tissue by the ramp excitation.
Association of Angiogenic and Inflammatory Markers with Power Doppler Ultrasound Vascularity Grade and DAS28-CRP in Early Rheumatoid Arthritis: A Comparative Analysis

Objective Upregulation of various proinflammatory and angiogenic mediators orchestrates the typical pathological synovial alterations in rheumatoid arthritis (RA). DAS28-CRP is commonly used for assessment of RA disease activity and power Doppler ultrasonography (PDUS) is an important modality for assessing synovial vascularity. This study was intended to look for the association of various inflammatory and angiogenic mediators, with respect to different PDUS vascularity grades and disease activity status, in early RA patients. Methods 50 early RA patients (<6 months disease duration) with either moderate or high disease activity and 30 healthy controls were included in this study. RA patients were subcategorized based on PDUS vascularity grading of wrist joints. Serum levels of proinflammatory cytokines [tumor necrosis factor-α (TNF- α); interleukin-6(IL-6)] and angiogenic markers [angiopoietin-1 and 2 (Ang-1 and Ang-2); vascular endothelial growth factor (VEGF) ] were measured and compared among different patient subgroups. Results Among 50 patients, 22 and 28 patients had moderate and high DAS28-CRP score, respectively. Patients with grade 3 PDUS score, even with moderate DAS value, showed a significant rise in Ang-1 (p<0.02), VEGF (p<0.008), Ang-2 (p <0.001), and TNF-α (p<0.005) level compared to grade 2 PDUS patients with high DAS values. Conclusion Higher serum level of angiogenic and inflammatory markers was noted among patients with moderate disease activity but with advanced PDUS vascularity (grade 3) in comparison to high disease activity group with less severe PDUS vascularity (grade 2). PDUS vascularity grading better reflects some markers of angiogenesis and inflammation, than composite disease activity index.
Association between the simultaneous decrease in the levels of soluble vascular cell adhesion molecule-1 and S100 protein and good neurological outcomes in cardiac arrest survivors

Objective This study aimed to determine whether simultaneous decreases in the serum levels of cell adhesion molecules (intracellular cell adhesion molecule-1 [ICAM-1], vascular cell adhesion molecule-1 [VCAM-1], and E-selectin) and S100 proteins within the first 24 hours after the return of spontaneous circulation were associated with good neurological outcomes in cardiac arrest survivors. Methods This retrospective observational study was based on prospectively collected data from a single emergency intensive care unit (ICU). Twenty-nine out-of-hospital cardiac arrest survivors who were admitted to the ICU for post-resuscitation care were enrolled. Blood samples were collected at 0 and 24 hours after ICU admission. According to the 6-month cerebral performance category (CPC) scale, the patients were divided into good (CPC 1 and 2, n=12) and poor (CPC 3 to 5, n=17) outcome groups. Results No difference was observed between the two groups in terms of the serum levels of ICAM-1, VCAM-1, E-selectin, and S100 at 0 and 24 hours. A simultaneous decrease in the serum levels of VCAM-1 and S100 as well as E-selectin and S100 was associated with good neurological outcomes. When other variables were adjusted, a simultaneous decrease in the serum levels of VCAM-1 and S100 was independently associated with good neurological outcomes (odds ratio, 9.285; 95% confidence interval, 1.073 to 80.318; P=0.043). Conclusion A simultaneous decrease in the serum levels of soluble VCAM-1 and S100 within the first 24 hours after the return of spontaneous circulation was associated with a good neurological outcome in out-of-hospital cardiac arrest survivors.
Fundamental bounds on learning performance in neural circuits

Significance We show how neural circuits can use additional connectivity to achieve faster and more precise learning. Biologically, internal synaptic noise imposes an optimal size of network for learning a given task. Above the optimal size, addition of neurons and synaptic connections starts to impede learning and task performance. Overall brain size may therefore be constrained by pressure to learn effectively with unreliable synapses and may explain why certain neurological learning deficits are associated with hyperconnectivity. Beneath this optimal size, apparently redundant connections are advantageous for learning. Such apparently redundant connections have recently been observed in several species and brain areas. How does the size of a neural circuit influence its learning performance? Larger brains tend to be found in species with higher cognitive function and learning ability. Intuitively, we expect the learning capacity of a neural circuit to grow with the number of neurons and synapses. We show how adding apparently redundant neurons and connections to a network can make a task more learnable. Consequently, large neural circuits can either devote connectivity to generating complex behaviors or exploit this connectivity to achieve faster and more precise learning of simpler behaviors. However, we show that in a biologically relevant setting where synapses introduce an unavoidable amount of noise, there is an optimal size of network for a given task. Above the optimal network size, the addition of neurons and synaptic connections starts to impede learning performance. This suggests that the size of brain circuits may be constrained by the need to learn efficiently with unreliable synapses and provides a hypothesis for why some neurological learning deficits are associated with hyperconnectivity. Our analysis is independent of specific learning rules and uncovers fundamental relationships between learning rate, task performance, network size, and intrinsic noise in neural circuits.
StarAlgo: A Squad Movement Planning Library for StarCraft using Monte Carlo Tree Search and Negamax

Real-Time Strategy (RTS) games have recently become a popular testbed for artificial intelligence research. They represent a complex adversarial domain providing a number of interesting AI challenges. There exists a wide variety of research-supporting software tools, libraries and frameworks for one RTS game in particular -- StarCraft: Brood War. These tools are designed to address various specific sub-problems, such as resource allocation or opponent modelling so that researchers can focus exclusively on the tasks relevant to them. We present one such tool -- a library called StarAlgo that produces plans for the coordinated movement of squads (groups of combat units) within the game world. StarAlgo library can solve the squad movement planning problem using one of two algorithms: Monte Carlo Tree Search Considering Durations (MCTSCD) and a slightly modified version of Negamax. We evaluate both the algorithms, compare them, and demonstrate their usage. The library is implemented as a static C++ library that can be easily plugged into most StarCraft AI bots.
Epipolar Geometry based Learning of Multi-view Depth and Ego-Motion from Monocular Sequences

Deep approaches to predict monocular depth and ego-motion have grown in recent years due to their ability to produce dense depth from monocular images. The main idea behind them is to optimize the photometric consistency over image sequences by warping one view into another, similar to direct visual odometry methods. One major drawback is that these methods infer depth from a single view, which might not effectively capture the relation between pixels. Moreover, simply minimizing the photometric loss does not ensure proper pixel correspondences, which is a key factor for accurate depth and pose estimations. In contrast, we propose a 2-view depth network to infer the scene depth from consecutive frames, thereby learning inter-pixel relationships. To ensure better correspondences, thereby better geometric understanding, we propose incorporating epipolar constraints to make the learning more geometrically sound. We use the Essential matrix obtained using Nistér's Five Point Algorithm, to enforce meaningful geometric constraints, rather than using it as training labels. This allows us to use lesser no. of trainable parameters compared to state-of-the-art methods. The proposed method results in better depth images and pose estimates, which capture the scene structure and motion in a better way. Such a geometrically constrained learning performs successfully even in cases where simply minimizing the photometric error would fail.
A Comprehensive Analysis of Swarming-based Live Streaming to Leverage Client Heterogeneity

Due to missing IP multicast support on an Internet scale, over-the-top media streams are delivered with the help of overlays as used by content delivery networks and their peer-to-peer (P2P) extensions. In this context, mesh/pull-based swarming plays an important role either as pure streaming approach or in combination with tree/push mechanisms. However, the impact of realistic client populations with heterogeneous resources is not yet fully understood. In this technical report, we contribute to closing this gap by mathematically analysing the most basic scheduling mechanisms latest deadline first (LDF) and earliest deadline first (EDF) in a continuous time Markov chain framework and combining them into a simple, yet powerful, mixed strategy to leverage inherent differences in client resources. The main contributions are twofold: (1) a mathematical framework for swarming on random graphs is proposed with a focus on LDF and EDF strategies in heterogeneous scenarios; (2) a mixed strategy, named SchedMix, is proposed that leverages peer heterogeneity. The proposed strategy, SchedMix is shown to outperform the other two strategies using different abstractions: a mean-field theoretic analysis of buffer probabilities, simulations of a stochastic model on random graphs, and a full-stack implementation of a P2P streaming system.
Image Super-Resolution via RL-CSC: When Residual Learning Meets Convolutional Sparse Coding

We propose a simple yet effective model for Single Image Super-Resolution (SISR), by combining the merits of Residual Learning and Convolutional Sparse Coding (RL-CSC). Our model is inspired by the Learned Iterative Shrinkage-Threshold Algorithm (LISTA). We extend LISTA to its convolutional version and build the main part of our model by strictly following the convolutional form, which improves the network's interpretability. Specifically, the convolutional sparse codings of input feature maps are learned in a recursive manner, and high-frequency information can be recovered from these CSCs. More importantly, residual learning is applied to alleviate the training difficulty when the network goes deeper. Extensive experiments on benchmark datasets demonstrate the effectiveness of our method. RL-CSC (30 layers) outperforms several recent state-of-the-arts, e.g., DRRN (52 layers) and MemNet (80 layers) in both accuracy and visual qualities. Codes and more results are available at this https URL.
Attacking Similarity-Based Link Prediction in Social Networks

Link prediction is one of the fundamental problems in computational social science. A particularly common means to predict existence of unobserved links is via structural similarity metrics, such as the number of common neighbors; node pairs with higher similarity are thus deemed more likely to be linked. However, a number of applications of link prediction, such as predicting links in gang or terrorist networks, are adversarial, with another party incentivized to minimize its effectiveness by manipulating observed information about the network. We offer a comprehensive algorithmic investigation of the problem of attacking similarity-based link prediction through link deletion, focusing on two broad classes of such approaches, one which uses only local information about target links, and another which uses global network information. While we show several variations of the general problem to be NP-Hard for both local and global metrics, we exhibit a number of well-motivated special cases which are tractable. Additionally, we provide principled and empirically effective algorithms for the intractable cases, in some cases proving worst-case approximation guarantees.
UAV Control in Close Proximities - Ceiling Effect on Battery Lifetime

With the recent developments in the unmanned aerial vehicles (UAV), it is expected them to interact and collaborate with their surrounding objects, other robots and people in order to wisely plan and execute particular tasks. Although these interaction operations are inherently challenging as compared to free-flight missions, they might bring diverse advantages. One of them is their basic aerodynamic interaction during the flight in close proximities which can result in a reduction of the controller effort. In this study, by collecting real-time data, we have observed that the current drawn by the battery can be decreased while flying very close to the surroundings with the help of the ceiling effect. For the first time, this phenomenon is analyzed in terms of battery lifetime degradation by using a simple full equivalent cycle counting method. Results show that cycling related effect on battery degradation can be reduced by a 15.77% if the UAV can utilize ceiling effect
Robust Subspace Clustering With Compressed Data

Dimension reduction is widely regarded as an effective way for decreasing the computation, storage, and communication loads of data-driven intelligent systems, leading to a growing demand for statistical methods that allow analysis (e.g., clustering) of compressed data. We therefore study in this paper a novel problem called compressive robust subspace clustering, which is to perform robust subspace clustering with the compressed data, and which is generated by projecting the original high-dimensional data onto a lower-dimensional subspace chosen at random. Given only the compressed data and sensing matrix, the proposed method, row space pursuit (RSP), recovers the authentic row space that gives correct clustering results under certain conditions. Extensive experiments show that RSP is distinctly better than the competing methods, in terms of both clustering accuracy and computational efficiency.
Profitable Bayesian implementation

In mechanism design theory, a designer would like to implement a desired social choice function which specifies her favorite outcome for each possible profile of all agents' types. Since agents' types are modelled as their private information, what the designer can do is to construct a mechanism and choose an outcome after observing a specific profile of agents' strategies. Traditionally, the designer has no way to adjust agents' types and hence may be in a dilemma in the sense that even if she is not satisfied with some outcome with low profit, she has to announce it because she must obey the mechanism designed by herself. In this paper, we generalize the mechanism design theory to a case where the designer can adjust the type distribution of agents, and propose a novel notion, \emph{i.e.}, profitable Bayesian implementation. After defining a series of notions such as adjusted types, optimal adjustment cost and profitable Bayesian implementability, we propose that the revelation principle does not hold in this generalized case. Finally, we construct an auction example to show that the designer can obtain an expected profit greater than the maximum profit that she can obtain in the traditional optimal auction, at the same time each agent's \emph{ex ante} expected profit is also increased.
On Checking Null Rank Conditions of Rational Matrices

In this paper we discuss possible numerical approaches to reliably check the rank condition rank $G(\lambda) = 0$ for a given rational matrix $G(\lambda)$ in terms of its descriptor system realization. For test purposes we employ functions available in the Control System Toolbox of MATLAB and the Descriptor System Tools (DSTOOLS) collection.