OpenRaiser/Intern-Atlas huggingface dataset:
Automated analysis of eye-tracker-based human-human interaction studies
Venue: arXiv cs.CV 2020
Year: 2020

Mobile eye-tracking systems have been available for about a decade now and are becoming increasingly popular in different fields of application, including marketing, sociology, usability studies and linguistics. While the user-friendliness and ergonomics of the hardware are developing at a rapid pace, the software for the analysis of mobile eye-tracking data in some points still lacks robustness and functionality. With this paper, we investigate which state-of-the-art computer vision algorithms may be used to automate the post-analysis of mobile eye-tracking data. For the case study in this paper, we focus on mobile eye-tracker recordings made during human-human face-to-face interactions. We compared two recent publicly available frameworks (YOLOv2 and OpenPose) to relate the gaze location generated by the eye-tracker to the head and hands visible in the scene camera data. In this paper we will show that the use of this single-pipeline framework provides robust results, which are both more accurate and faster than previous work in the field. Moreover, our approach does not rely on manual interventions during this process.
Training Restricted Boltzmann Machines with Binary Synapses using the Bayesian Learning Rule
Venue: arXiv cs.LG 2020
Year: 2020

Restricted Boltzmann machines (RBMs) with low-precision synapses are much appealing with high energy efficiency. However, training RBMs with binary synapses is challenging due to the discrete nature of synapses. Recently Huang proposed one efficient method to train RBMs with binary synapses by using a combination of gradient ascent and the message passing algorithm under the variational inference framework. However, additional heuristic clipping operation is needed. In this technical note, inspired from Huang's work , we propose one alternative optimization method using the Bayesian learning rule, which is one natural gradient variational inference method. As opposed to Huang's method, we update the natural parameters of the variational symmetric Bernoulli distribution rather than the expectation parameters. Since the natural parameters take values in the entire real domain, no additional clipping is needed. Interestingly, the algorithm in \cite{huang2019data} could be viewed as one first-order approximation of the proposed algorithm, which justifies its efficacy with heuristic clipping.
How low can you go? Privacy-preserving people detection with an omni-directional camera
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we use a ceiling-mounted omni-directional camera to detect people in a room. This can be used as a sensor to measure the occupancy of meeting rooms and count the amount of flex-desk working spaces available. If these devices can be integrated in an embedded low-power sensor, it would form an ideal extension of automated room reservation systems in office environments. The main challenge we target here is ensuring the privacy of the people filmed. The approach we propose is going to extremely low image resolutions, such that it is impossible to recognise people or read potentially confidential documents. Therefore, we retrained a single-shot low-resolution person detection network with automatically generated ground truth. In this paper, we prove the functionality of this approach and explore how low we can go in resolution, to determine the optimal trade-off between recognition accuracy and privacy preservation. Because of the low resolution, the result is a lightweight network that can potentially be deployed on embedded hardware. Such embedded implementation enables the development of a decentralised smart camera which only outputs the required meta-data (i.e. the number of persons in the meeting room).
Green Lighting ML: Confidentiality, Integrity, and Availability of Machine Learning Systems in Deployment
Venue: arXiv cs.CY 2020
Year: 2020

Security and ethics are both core to ensuring that a machine learning system can be trusted. In production machine learning, there is generally a hand-off from those who build a model to those who deploy a model. In this hand-off, the engineers responsible for model deployment are often not privy to the details of the model and thus, the potential vulnerabilities associated with its usage, exposure, or compromise. Techniques such as model theft, model inversion, or model misuse may not be considered in model deployment, and so it is incumbent upon data scientists and machine learning engineers to understand these potential risks so they can communicate them to the engineers deploying and hosting their models. This is an open problem in the machine learning community and in order to help alleviate this issue, automated systems for validating privacy and security of models need to be developed, which will help to lower the burden of implementing these hand-offs and increasing the ubiquity of their adoption.
A Systematic Review on Context-Aware Recommender Systems using Deep Learning and Embeddings
Venue: arXiv cs.IR 2020
Year: 2020

Recommender Systems are tools that improve how users find relevant information in web systems, so they do not face too much information. In order to generate better recommendations, the context of information should be used in the recommendation process. Context-Aware Recommender Systems were created, accomplishing state-of-the-art results and improving traditional recommender systems. There are many approaches to build recommender systems, and two of the most prominent advances in area have been the use of Embeddings to represent the data in the recommender system, and the use of Deep Learning architectures to generate the recommendations to the user. A systematic review adopts a formal and systematic method to perform a bibliographic review, and it is used to identify and evaluate all the research in certain area of study, by analyzing the relevant research published. A systematic review was conducted to understand how the Deep Learning and Embeddings techniques are being applied to improve Context-Aware Recommender Systems. We summarized the architectures that are used to create those and the domains that they are used.
MO-PaDGAN: Generating Diverse Designs with Multivariate Performance Enhancement
Venue: arXiv cs.LG 2020
Year: 2020

Deep generative models have proven useful for automatic design synthesis and design space exploration. However, they face three challenges when applied to engineering design: 1) generated designs lack diversity, 2) it is difficult to explicitly improve all the performance measures of generated designs, and 3) existing models generally do not generate high-performance novel designs, outside the domain of the training data. To address these challenges, we propose MO-PaDGAN, which contains a new Determinantal Point Processes based loss function for probabilistic modeling of diversity and performances. Through a real-world airfoil design example, we demonstrate that MO-PaDGAN expands the existing boundary of the design space towards high-performance regions and generates new designs with high diversity and performances exceeding training data.
Targeting the Benchmark: On Methodology in Current Natural Language Processing Research
Venue: arXiv cs.CL 2020
Year: 2020

It has become a common pattern in our field: One group introduces a language task, exemplified by a dataset, which they argue is challenging enough to serve as a benchmark. They also provide a baseline model for it, which then soon is improved upon by other groups. Often, research efforts then move on, and the pattern repeats itself. What is typically left implicit is the argumentation for why this constitutes progress, and progress towards what. In this paper, we try to step back for a moment from this pattern and work out possible argumentations and their parts.
Client Adaptation improves Federated Learning with Simulated Non-IID Clients
Venue: arXiv cs.LG 2020
Year: 2020

We present a federated learning approach for learning a client adaptable, robust model when data is non-identically and non-independently distributed (non-IID) across clients. By simulating heterogeneous clients, we show that adding learned client-specific conditioning improves model performance, and the approach is shown to work on balanced and imbalanced data set from both audio and image domains. The client adaptation is implemented by a conditional gated activation unit and is particularly beneficial when there are large differences between the data distribution for each client, a common scenario in federated learning.
Predicting Court Decisions for Alimony: Avoiding Extra-legal Factors in Decision made by Judges and Not Understandable AI Models
Venue: arXiv cs.AI 2020
Year: 2020

The advent of machine learning techniques has made it possible to obtain predictive systems that have overturned traditional legal practices. However, rather than leading to systems seeking to replace humans, the search for the determinants in a court decision makes it possible to give a better understanding of the decision mechanisms carried out by the judge. By using a large amount of court decisions in matters of divorce produced by French jurisdictions and by looking at the variables that allow to allocate an alimony or not, and to define its amount, we seek to identify if there may be extra-legal factors in the decisions taken by the judges. From this perspective, we present an explainable AI model designed in this purpose by combining a classification with random forest and a regression model, as a complementary tool to existing decision-making scales or guidelines created by practitioners.
Predictability of real temporal networks
Venue: arXiv cs.SI 2020
Year: 2020

Links in most real networks often change over time. Such temporality of links encodes the ordering and causality of interactions between nodes and has a profound effect on network dynamics and function. Empirical evidences have shown that the temporal nature of links in many real-world networks is not random. Nonetheless, it is challenging to predict temporal link patterns while considering the entanglement between topological and temporal link patterns. Here we propose an entropy-rate based framework, based on combined topological-temporal regularities, for quantifying the predictability of any temporal network. We apply our framework on various model networks, demonstrating that it indeed captures the intrinsic topological-temporal regularities whereas previous methods considered only temporal aspects. We also apply our framework on 18 real networks of different types and determine their predictability. Interestingly, we find that for most real temporal networks, despite the greater complexity of predictability brought by the increase in dimension the combined topological-temporal predictability is higher than the temporal predictability. Our results demonstrate the necessity of incorporating both temporal and topological aspects of networks in order to improve predictions of dynamical processes.
A Consent Model for Blockchain-based Distributed Data Sharing Platforms
Venue: arXiv cs.DC 2020
Year: 2020

In modern healthcare systems, being able to share electronic health records is crucial for providing quality care and for enabling a larger spectrum of health services. Health data sharing is dependent on obtaining individual consent which, in turn, is hindered by a lack of resources. To this extend, blockchain-based platforms facilitate data sharing by inherently creating a trusted distributed network of users. These users are enabled to share their data without depending on the time and resources of specific players (such as the health services). In blockchain-based platforms, data governance mechanisms become very important due to the need to specify and monitor data sharing and data use conditions. In this paper, we present a blockchain-based data sharing consent model for access control over individual health data. We use smart contracts to dynamically represent the individual consent over health data and to enable data requesters to search and access them. The dynamic consent model extends upon two ontologies: the Data Use Ontology (DUO) which models the individual consent of users and the Automatable Discovery and Access Matrix (ADA-M) which describes queries from data requesters. We deploy the model on Ethereum blockchain and evaluate different data sharing scenarios. The contribution of this paper is to create an individual consent model for health data sharing platforms. Such a model guarantees that individual consent is respected and that there is accountability for all the participants in the data sharing platform. The evaluation of our solution indicates that such a data sharing model provides a flexible approach to decide how the data is used by data requesters. Our experimental evaluation shows that the proposed model is efficient and adapts to personalized access control policies in data sharing.
Attention or memory? Neurointerpretable agents in space and time
Venue: arXiv cs.AI 2020
Year: 2020

In neuroscience, attention has been shown to bidirectionally interact with reinforcement learning (RL) processes. This interaction is thought to support dimensionality reduction of task representations, restricting computations to relevant features. However, it remains unclear whether these properties can translate into real algorithmic advantages for artificial agents, especially in dynamic environments. We design a model incorporating a self-attention mechanism that implements task-state representations in semantic feature-space, and test it on a battery of Atari games. To evaluate the agent's selective properties, we add a large volume of task-irrelevant features to observations. In line with neuroscience predictions, self-attention leads to increased robustness to noise compared to benchmark models. Strikingly, this self-attention mechanism is general enough, such that it can be naturally extended to implement a transient working-memory, able to solve a partially observable maze task. Lastly, we highlight the predictive quality of attended stimuli. Because we use semantic observations, we can uncover not only which features the agent elects to base decisions on, but also how it chooses to compile more complex, relational features from simpler ones. These results formally illustrate the benefits of attention in deep RL and provide evidence for the interpretability of self-attention mechanisms.
Real-time Embedded Person Detection and Tracking for Shopping Behaviour Analysis
Venue: arXiv cs.CV 2020
Year: 2020

Shopping behaviour analysis through counting and tracking of people in shop-like environments offers valuable information for store operators and provides key insights in the stores layout (e.g. frequently visited spots). Instead of using extra staff for this, automated on-premise solutions are preferred. These automated systems should be cost-effective, preferably on lightweight embedded hardware, work in very challenging situations (e.g. handling occlusions) and preferably work real-time. We solve this challenge by implementing a real-time TensorRT optimized YOLOv3-based pedestrian detector, on a Jetson TX2 hardware platform. By combining the detector with a sparse optical flow tracker we assign a unique ID to each customer and tackle the problem of loosing partially occluded customers. Our detector-tracker based solution achieves an average precision of 81.59% at a processing speed of 10 FPS. Besides valuable statistics, heat maps of frequently visited spots are extracted and used as an overlay on the video stream.
AI Assisted Apparel Design
Venue: arXiv cs.CV 2020
Year: 2020

Fashion is a fast-changing industry where designs are refreshed at large scale every season. Moreover, it faces huge challenge of unsold inventory as not all designs appeal to customers. This puts designers under significant pressure. Firstly, they need to create innumerous fresh designs. Secondly, they need to create designs that appeal to customers. Although we see advancements in approaches to help designers analyzing consumers, often such insights are too many. Creating all possible designs with those insights is time consuming. In this paper, we propose a system of AI assistants that assists designers in their design journey. The proposed system assists designers in analyzing different selling/trending attributes of apparels. We propose two design generation assistants namely Apparel-Style-Merge and Apparel-Style-Transfer. Apparel-Style-Merge generates new designs by combining high level components of apparels whereas Apparel-Style-Transfer generates multiple customization of apparels by applying different styles, colors and patterns. We compose a new dataset, named DeepAttributeStyle, with fine-grained annotation of landmarks of different apparel components such as neck, sleeve etc. The proposed system is evaluated on a user group consisting of people with and without design background. Our evaluation result demonstrates that our approach generates high quality designs that can be easily used in fabrication. Moreover, the suggested designs aid to the designers creativity.
StyPath: Style-Transfer Data Augmentation For Robust Histology Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within) pathology labs, coupled with the gradual fading of archival sections. Variations in stain colors and intensities can make tissue evaluation difficult for pathologists, ultimately affecting their ability to describe relevant morphological features. Being able to accurately predict the AMR status based on kidney histology images is crucial for improving patient treatment and care. We propose a novel pipeline to build robust deep neural networks for AMR classification based on StyPath, a histological data augmentation technique that leverages a light weight style-transfer algorithm as a means to reduce sample-specific bias. Each image was generated in 1.84 +- 0.03 seconds using a single GTX TITAN V gpu and pytorch, making it faster than other popular histological data augmentation techniques. We evaluated our model using a Monte Carlo (MC) estimate of Bayesian performance and generate an epistemic measure of uncertainty to compare both the baseline and StyPath augmented models. We also generated Grad-CAM representations of the results which were assessed by an experienced nephropathologist; we used this qualitative analysis to elucidate on the assumptions being made by each model. Our results imply that our style-transfer augmentation technique improves histological classification performance (reducing error from 14.8% to 11.5%) and generalization ability.
Design of Puncturing for Length-Compatible Polar Codes Using Differential Evolution
Venue: arXiv cs.IT 2020
Year: 2020

This paper presents a puncturing technique to design length-compatible polar codes. The punctured bits are identified with the help of differential evolution (DE). A DE-based optimization framework is developed where the sum of the bit-error-rate (BER) values of the information bits is minimized. We identify a set of bits which can be avoided for puncturing in the case of additive white Gaussian noise (AWGN) channels. This reduces the size of the candidate puncturing patterns. Simulation results confirm the superiority of the proposed technique over other state-of-the-art puncturing methods.
Learning dynamical systems from data: a simple cross-validation perspective
Venue: arXiv cs.LG 2020
Year: 2020

Regressing the vector field of a dynamical system from a finite number of observed states is a natural way to learn surrogate models for such systems. We present variants of cross-validation (Kernel Flows \cite{Owhadi19} and its variants based on Maximum Mean Discrepancy and Lyapunov exponents) as simple approaches for learning the kernel used in these emulators.
A Benchmark for Inpainting of Clothing Images with Irregular Holes
Venue: arXiv cs.CV 2020
Year: 2020

Fashion image understanding is an active research field with a large number of practical applications for the industry. Despite its practical impacts on intelligent fashion analysis systems, clothing image inpainting has not been extensively examined yet. For that matter, we present an extensive benchmark of clothing image inpainting on well-known fashion datasets. Furthermore, we introduce the use of a dilated version of partial convolutions, which efficiently derive the mask update step, and empirically show that the proposed method reduces the required number of layers to form fully-transparent masks. Experiments show that dilated partial convolutions (DPConv) improve the quantitative inpainting performance when compared to the other inpainting strategies, especially it performs better when the mask size is 20% or more of the image. \keywords{image inpainting, fashion image understanding, dilated convolutions, partial convolutions
The Trade-Offs of Private Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning models leak information about their training data every time they reveal a prediction. This is problematic when the training data needs to remain private. Private prediction methods limit how much information about the training data is leaked by each prediction. Private prediction can also be achieved using models that are trained by private training methods. In private prediction, both private training and private prediction methods exhibit trade-offs between privacy, privacy failure probability, amount of training data, and inference budget. Although these trade-offs are theoretically well-understood, they have hardly been studied empirically. This paper presents the first empirical study into the trade-offs of private prediction. Our study sheds light on which methods are best suited for which learning setting. Perhaps surprisingly, we find private training methods outperform private prediction methods in a wide range of private prediction settings.
SGQuant: Squeezing the Last Bit on Graph Neural Networks with Specialized Quantization
Venue: arXiv cs.LG 2020
Year: 2020

With the increasing popularity of graph-based learning, Graph Neural Networks (GNNs) win lots of attention from the research and industry field because of their high accuracy. However, existing GNNs suffer from high memory footprints (e.g., node embedding features). This high memory footprint hurdles the potential applications towards memory-constrained devices, such as the widely-deployed IoT devices. To this end, we propose a specialized GNN quantization scheme, SGQuant, to systematically reduce the GNN memory consumption. Specifically, we first propose a GNN-tailored quantization algorithm design and a GNN quantization fine-tuning scheme to reduce memory consumption while maintaining accuracy. Then, we investigate the multi-granularity quantization strategy that operates at different levels (components, graph topology, and layers) of GNN computation. Moreover, we offer an automatic bit-selecting (ABS) to pinpoint the most appropriate quantization bits for the above multi-granularity quantizations. Intensive experiments show that SGQuant can effectively reduce the memory footprint from 4.25x to 31.9x compared with the original full-precision GNNs while limiting the accuracy drop to 0.4% on average.