Miracle-dz/UniFAR-FTUs huggingface dataset:
Broadcasting Convolutional Network

While convolutional neural networks (CNNs) are widely used for handling spatio-temporal scenes, there exist limitations in reasoning relations among spatial features caused by their inherent structures, which have been issued consistently in many studies. In this paper, we propose Broadcasting Convolutional Networks (BCN) that allow global receptive fields to share spatial information. BCNs are simple network modules that collect effective spatial features, embed location informations and broadcast them to the entire feature maps without much additional computational cost. This method gains great improvements in feature localization problems through efficiently extending the receptive fields, and can easily be implemented within any structure of CNNs. We further utilize BCN to propose Multi-Relational Networks (multiRN) that greatly improve existing Relation Networks (RNs). In pixel-based relation reasoning problems, multiRN with BCNs implanted extends the concept of `pairwise relations' from conventional RNs to `multiple relations' by relating each object with multiple objects at once and not in pairs. This yields in O(n) complexity for n number of objects, which is a vast computational gain from RNs that take O(n^2). Through experiments, BCNs are proven for their usability on relation reasoning problems, which is due from their efficient handlings of spatial information.
DoShiCo: a Domain Shift Challenge for Control

Training deep neural control networks end-to-end for real-world applications typically requires big demonstration datasets in the real world or big sets consisting of a large variety of realistic and closely related 3D CAD models. These real or virtual data should, moreover, have very similar characteristics to the conditions expected at test time. These stringent requirements and the time consuming data collection processes that they entail, are probably the most important impediment that keeps deep neural policies from being deployed in real-world applications. Therefore, in this work we advocate an alternative approach, where instead of avoiding any domain shift by carefully selecting the training data, the goal is to learn a policy that can cope with it. To this end, we propose a new challenge: to train a model in very basic synthetic environments, far from realistic, in a way that it can fly in more realistic environments as well as take the control decisions on real-world data. We collected a benchmark dataset and implemented a baseline method, exploiting depth prediction as an auxiliary task to help overcome the domain shift. Even though the policy is trained in very basic environments, it can learn to fly in a very different realistic simulated environment. It is even capable to compete and in some cases outperform a policy trained in the more realistic environment when testing on real-world data.
BOPS, Not FLOPS! A New Metric, Measuring Tool, and Roofline Performance Model For Datacenter Computing

The past decades witness FLOPS (Floating-point Operations per Second), as an important computation-centric performance metric, guides computer architecture evolution, bridges hardware and software co-design, and provides quantitative performance number for system optimization. However, for emerging datacenter computing (in short, DC) workloads, such as internet services or big data analytics, previous work reports on the modern CPU architecture that the average proportion of floating-point instructions only takes 1% and the average FLOPS efficiency is only 0.1%, while the average CPU utilization is high as 63%. These contradicting performance numbers imply that FLOPS is inappropriate for evaluating DC computer systems. To address the above issue, we propose a new computation-centric metric BOPS (Basic OPerations per Second). In our definition, Basic Operations include all of arithmetic, logical, comparing and array addressing operations for integer and floating point. BOPS is the average number of BOPs (Basic OPerations) completed each second. To that end, we present a dwarf-based measuring tool to evaluate DC computer systems in terms of our new metrics. On the basis of BOPS, also we propose a new roofline performance model for DC computing. Through the experiments, we demonstrate that our new metrics--BOPS, measuring tool, and new performance model indeed facilitate DC computer system design and optimization.
Achievable Rate of Relay Assisted Cooperative O-NOMA under Rician Fading Channels

The achievable rate of an opportunistic nonorthogonal multiple access (O-NOMA) based cooperative relaying system (CRS) with channel state information (CSI) available at the transmitter end is analyzed under Rician fading channels. O-NOMA-based CRS achieves better achievable rate than CNOMA-based CRS and results are verified through Monte Carlo simulations. Keywords—Non-orthogonal Multiple Access, Rician Fading Channels, Average Achievable Rate
Improving Smiling Detection with Race and Gender Diversity

Recent progress in deep learning has been accompanied by a growing concern for whether models are fair for users, with equally good performance across different demographics. In computer vision research, such questions are relevant to face detection and the related task of face attribute detection, among others. We measure race and gender inclusion in the context of smiling detection, and introduce a method for improving smiling detection across demographic groups. Our method introduces several modifications over existing detection methods, leveraging twofold transfer learning to better model facial diversity. Results show that this technique improves accuracy against strong baselines for most demographic groups as well as overall. Our best-performing model defines a new state-of-the-art for smiling detection, reaching 91% on the Faces of the World dataset. The accompanying multi-head diversity classifier also defines a new state-of-the-art for gender classification, reaching 93.87% on the Faces of the World dataset. This research demonstrates the utility of modeling race and gender to improve a face attribute detection task, using a twofold transfer learning framework that allows for privacy towards individuals in a target dataset.
Using Visual Saliency to Improve Human Detection with Convolutional Networks

In this paper, we demonstrate an approach based on visual saliency for detection of humans. Using Deep Multi-Layer Network [1], we find the saliency maps of an image having humans, multiply with the input image and fed to Convolutional Neural Network (CNN). For detection purpose, we train DetectNet on prepared two challenging datasets - Penn-Fudan Dataset and TudBrussels Benchmark. After training, the network learns the mid and high-level features of a human body. We show the effectiveness of our approach on both the tasks and report state-of-the-art performance on PennFudan Dataset with the detection accuracy of 91.4% and achieve average miss-rate of 53% on the TudBrussels Benchmark.
Visualizing operations on temporal data

An interactive data browsing environment designed specifically to support temporal manipulations is described. A formal system of temporal operations is combined with a visual system for interactive temporal data display. The basic temporal construct is a time line. For each of the basic time-line operations, the effects of each operator, including exceptions and boundary conditions, are defined. For each formal definition, a precise visual analog of the operation is given. A time-line browser, an editor for time lines, which implements this formalism is described and applied to biomedical data (e.g. diabetes patient data). <<ETX>>
Towards Semantic SLAM: Points, Planes and Objects

Simultaneous Localization And Mapping (SLAM) is a fundamental problem in mobile robotics. Semantic SLAM is an effort to build meaningful map representations that not only provide rich information about the environment but also aid in camera localization. This work proposes a method for representing generic objects using quadrics that allows seamless integration in a SLAM framework, with additional dominant planar structure modeled as infinite planes. Experiments show the proposed points-planes-quadrics representation can easily incorporate Manhattan and object affordance constraints, greatly improving camera localization and leading to semantically meaningful maps.
Manipulation Planning to Keep an Object Stable under a Sequence of External Forces

We present a planner for a robot to keep an object stable under a sequence of external forces. We assume a robot with multiple manipulators, with a gripper at the end of each manipulator. We make two key contributions: First, given an external force, we propose a method to check the feasibility of gripper placements on the object to resist the force. This method takes into account both the manipulator joint torque limits and the force/torque limits of the grippers-object system. Second, given a sequence of external forces, we propose a planner that generates a sequence of gripper placements and manipulator configurations, along with the motion to move between these configurations. The planner minimizes the number of different gripper placements required to resist the forces, resulting in efficient planning and execution. We perform experiments to verify the predictions of our feasibility checking method and to measure the performance of our planner.
REOH: Runtime Energy Optimization for Heterogeneous Systems

Significant efforts have been devoted to choosing the best configuration of a computing system to run an application energy efficiently. However, available tuning approaches mainly focus on homogeneous systems and are inextensible for heterogeneous systems which include several components (e.g., CPUs, GPUs) with different architectures. This study proposes a holistic tuning approach called REOH, based on probabilistic models to predict the most energy-efficient configuration (i.e., which platform and its setting) of a heterogeneous system for running a given application. Based on the computation and communication patterns from Berkeley dwarfs, we conduct experiments to devise the training data set that covers a wide range of application patterns and characteristics. Validating the REOH approach on heterogeneous systems including CPUs and GPUs shows that the energy consumption by the REOH approach is only one third the energy consumption by the previous (homogeneous) approaches in the best cases and close to the optimal energy consumption by the brute-force approach. Based on the REOH approach, we develop a open-source energy-optimizing runtime framework for selecting an energy efficient configuration of a heterogeneous system for a given application at runtime.
Lane Detection for Intelligent Vehicles in Challenging Scenarios

Various image processing techniques and geometric models have been applied in vision based lane detection subsystems of intelligent vehicles and Advanced Driver Assistance Systems (ADAS). However, challenging conditions such as strong shadows, occlusions, eroded markings, high curvatures are ongoing issues in this topic. In this paper, a novel lane extraction method based on symmetrical local threshold is proposed with the hyperbola road model. Optimal model fit of lane distribution and hyperbola-pairs are estimated with RANdom SAmple Consensus (RANSAC) technique. The system computes the road geometry, vehicle position and direction both on urban road and country roads. Results have been presented on images with strong shadows, high curvature and eroded markings. Road and vehicle parameters computed on synthetic images are compared with ground truth information to show the accuracy of the proposed method.
Improving Brain Storm Optimization Algorithm via Simplex Search

Through modeling human's brainstorming process, the brain storm optimization (BSO) algorithm has become a promising population based evolution algorithm. However, BSO is often good at global exploration but not good enough at local exploitation, just like most global optimization algorithms. In this paper, the Nelder-Mead's Simplex (NMS) method is adopted in a simple version of BSO. Our goal is to combine BSO's exploration ability and NMS's exploitation ability together, and develop an enhanced BSO via a better balance between global exploration and local exploitation. Large number of experimental results are reported, and the proposed algorithm is shown to perform better than both BSO and NMS.
Predicting the Future Popularity of Images on Social Networks

This paper addresses the problem of predicting future image popularity on social networks by considering the changes of image popularity over time. For this problem, we collect information about an image within an hour of upload and keep track of its popularity for one month to predict its future popularity (e.g., after a day, a week, a month). For the prediction of popularity, we employ three features: social context (i.e., user information), the image's semantics and the image's early popularity. We propose a novel approach to extract the semantic of images, based on well established natural language processing and clustering techniques. Using a Gaussian Naive Bayes classifier, we predict the future popularity of images using such social context, image semantics, and early popularity. The results show that the accuracy of the classifier reaches 90% on average in predicting future popularity; image semantics is the only feature that increases popularity predictions accuracy along the timeline.
Learning Loss for Knowledge Distillation with Conditional Adversarial Networks

There is an increasing interest on accelerating neural networks for real-time applications. We study the student-teacher strategy, in which a small and fast student network is trained with the auxiliary information provided by a large and accurate teacher network. We use conditional adversarial networks to learn the loss function to transfer knowledge from teacher to student. The proposed method is particularly effective for relatively small student networks. Moreover, experimental results show the effect of network size when the modern networks are used as student. We empirically study trade-off between inference time and classification accuracy, and provide suggestions on choosing a proper student.
HypDB: Detect, Explain And Resolve Bias in OLAP

On line analytical processing (OLAP) is an essential element of decision-support systems. OLAP tools provide insights and understanding needed for improved decision making. However, the answers to OLAP queries can be biased and lead to perplexing and incorrect insights. In this paper, we propose HypDB, a system to detect, explain, and to resolve bias in decision-support queries. We give a simple definition of a \emph{biased query}, which performs a set of independence tests on the data to detect bias. We propose a novel technique that gives explanations for bias, thus assisting an analyst in understanding what goes on. Additionally, we develop an automated method for rewriting a biased query into an unbiased query, which shows what the analyst intended to examine. In a thorough evaluation on several real datasets we show both the quality and the performance of our techniques, including the completely automatic discovery of the revolutionary insights from a famous 1973 discrimination case.
Removing scanner biases using Generative Adversarial Networks

Magnetic Resonance Imaging (MRI) of the brain has been used to investigate a wide range of neurological disorders, but data acquisition can be expensive, time-consuming, and inconvenient. Multi-site studies present a valuable opportunity to advance research by pooling data in order to increase sensitivity and statistical power. However images derived from MRI are susceptible to both obvious and non-obvious differences between sites which can introduce bias and subject variance, and so reduce statistical power. To rectify these differences, we propose a data driven approach using a deep learning architecture known as generative adversarial networks (GANs). GANs learn to estimate two distributions, and can then be used to transform examples from one distribution into the other distribution. Here we transform T1-weighted brain images collected from two different sites into MR images from the same site. We evaluate whether our model can reduce site-specific differences without loss of information related to gender (male, female) or clinical diagnosis (schizophrenia, bipolar disorder, healthy). When trained appropriately, our model is able to normalise imaging sets to a common scanner set with less information loss compared to current approaches. An important advantage is our method can be treated as a black box that does not require any knowledge of the sources of bias but only needs at least two distinct imaging sets.
Cross Corpus Speech Emotion Classification- An Effective Transfer Learning Technique

Cross-corpus speech emotion recognition can be a useful transfer learning technique to build a robust speech emotion recognition system by leveraging information from various speech datasets - cross-language and cross-corpus. However, more research needs to be carried out to understand the effective operating scenarios of cross-corpus speech emotion recognition, especially with the utilization of the powerful deep learning techniques. In this paper, we use five different corpora of three different languages to investigate the cross-corpus and cross-language emotion recognition using Deep Belief Networks (DBNs). Experimental results demonstrate that DBNs with generalization power offers better accuracy than a discriminative method based on Sparse Auto Encoder and SVM. Results also suggest that using a large number of languages for training and using a small fraction of target data in training can significantly boost accuracy compared to using the same language for training and testing.
Efficient Use of Limited-Memory Resources to Accelerate Linear Learning

We propose a generic algorithmic building block to accelerate training of machine learning models on heterogenous compute systems. The scheme allows to efficiently employ compute accelerators such as GPUs and FPGAs for the training of large-scale machine learning models, when the training data exceeds their memory capacity. Also, it provides adaptivity to any system's memory hierarchy in terms of size and processing speed. Our technique builds upon primal-dual coordinate methods, and uses duality gap information to dynamically decide which part of the data should be made available for fast processing. We provide a strong theoretical motivation for our gap-based selection scheme and provide an efficient practical implementation thereof. To illustrate the power of our approach we demonstrate its performance for training of generalized linear models on large scale datasets exceeding the memory size of a modern GPU, showing an order-of-magnitude speedup over existing approaches.
Group Fairness in Multiwinner Voting

We study multiwinner voting problems when there is an additional requirement that the selected committee should be fair with respect to attributes such as gender, ethnicity, or political parties. Every setting of an attribute gives rise to a group, and the goal is to ensure that each group is neither over nor under represented in the selected committee. Prior work has largely focused on designing specialized score functions that lead to a precise level of representation with respect to disjoint attributes (e.g., only political affiliation). Here we propose a general algorithmic framework that allows the use of any score function and can guarantee flexible notions of fairness with respect to multiple, non-disjoint attributes (e.g., political affiliation and gender). Technically, we study the complexity of this constrained multiwinner voting problem subject to group-fairness constraints for monotone submodular score functions. We present approximation algorithms and hardness of approximation results for various attribute set structures and score functions.
Deep Net Triage: Assessing the Criticality of Network Layers by Structural Compression

Deep network compression seeks to reduce the number of parameters in the network while maintaining a certain level of performance. Deep network distillation seeks to train a smaller network that matches soft-max performance of a larger network. While both regimes have led to impressive performance for their respective goals, neither provide insight into the importance of a given layer in the original model, which is useful if we are to improve our understanding of these highly parameterized models. In this paper, we present the concept of deep net triage, which individually assesses small blocks of convolution layers to understand their collective contribution to the overall performance, which we call \emph{criticality}. We call it triage because we assess this criticality by answering the question: what is the impact to the health of the overall network if we compress a block of layers into a single layer. We propose a suite of triage methods and compare them on problem spaces of varying complexity. We ultimately show that, across these problem spaces, deep net triage is able to indicate the of relative importance of different layers. Surprisingly, our local structural compression technique also leads to an improvement in overall accuracy when the final model is fine-tuned globally.
Learning Differentially Private Language Models Without Losing Accuracy

We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees without sacrificing predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes"large step"updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.
Deep Cascade Multi-task Learning for Slot Filling in Chinese E-commerce Shopping Guide Assistant

Slot filling is a critical task in natural language understanding (NLU) for dialog systems. State-of-the-art solutions regard it as a sequence label- ing task and adopt BiLSTM-CRF models. While BiLSTM-CRF models works relatively well on standard datasets it faces challenges in Chinese E-commerce slot filling due to more informative slot labels and richer expressions. In this paper, we propose a deep multi-task learning model with cascade and residual connections. Experimental results show that our framework not only achieves competitive performance with state-of-the-arts on a standard dataset, but also significantly outperforms strong baselines by a substantial gain of 14.6% on a Chinese E-commerce dataset.