OpenRaiser/Intern-Atlas huggingface dataset:
Robust Federated Learning with Noisy Labels
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning is a paradigm that enables local devices to jointly train a server model while keeping the data decentralized and private. In federated learning, since local data are collected by clients, it is hardly guaranteed that the data are correctly annotated. Although a lot of studies have been conducted to train the networks robust to these noisy data in a centralized setting, these algorithms still suffer from noisy labels in federated learning. Compared to the centralized setting, clients' data can have different noise distributions due to variations in their labeling systems or background knowledge of users. As a result, local models form inconsistent decision boundaries and their weights severely diverge from each other, which are serious problems in federated learning. To solve these problems, we introduce a novel federated learning scheme that the server cooperates with local models to maintain consistent decision boundaries by interchanging class-wise centroids. These centroids are central features of local data on each device, which are aligned by the server every communication round. Updating local models with the aligned centroids helps to form consistent decision boundaries among local models, although the noise distributions in clients' data are different from each other. To improve local model performance, we introduce a novel approach to select confident samples that are used for updating the model with given labels. Furthermore, we propose a global-guided pseudo-labeling method to update labels of unconfident samples by exploiting the global model. Our experimental results on the noisy CIFAR-10 dataset and the Clothing1M dataset show that our approach is noticeably effective in federated learning with noisy labels.
A journey in ESN and LSTM visualisations on a language task
Venue: arXiv cs.NE 2020
Year: 2020

Echo States Networks (ESN) and Long-Short Term Memory networks (LSTM) are two popular architectures of Recurrent Neural Networks (RNN) to solve machine learning task involving sequential data. However, little have been done to compare their performances and their internal mechanisms on a common task. In this work, we trained ESNs and LSTMs on a Cross-Situationnal Learning (CSL) task. This task aims at modelling how infants learn language: they create associations between words and visual stimuli in order to extract meaning from words and sentences. The results are of three kinds: performance comparison, internal dynamics analyses and visualization of latent space. (1) We found that both models were able to successfully learn the task: the LSTM reached the lowest error for the basic corpus, but the ESN was quicker to train. Furthermore, the ESN was able to outperform LSTMs on datasets more challenging without any further tuning needed. (2) We also conducted an analysis of the internal units activations of LSTMs and ESNs. Despite the deep differences between both models (trained or fixed internal weights), we were able to uncover similar inner mechanisms: both put emphasis on the units encoding aspects of the sentence structure. (3) Moreover, we present Recurrent States Space Visualisations (RSSviz), a method to visualize the structure of latent state space of RNNs, based on dimension reduction (using UMAP). This technique enables us to observe a fractal embedding of sequences in the LSTM. RSSviz is also useful for the analysis of ESNs (i) to spot difficult examples and (ii) to generate animated plots showing the evolution of activations across learning stages. Finally, we explore qualitatively how the RSSviz could provide an intuitive visualisation to understand the influence of hyperparameters on the reservoir dynamics prior to ESN training.
NICER: Aesthetic Image Enhancement with Humans in the Loop
Venue: arXiv cs.HC 2020
Year: 2020

Fully- or semi-automatic image enhancement software helps users to increase the visual appeal of photos and does not require in-depth knowledge of manual image editing. However, fully-automatic approaches usually enhance the image in a black-box manner that does not give the user any control over the optimization process, possibly leading to edited images that do not subjectively appeal to the user. Semi-automatic methods mostly allow for controlling which pre-defined editing step is taken, which restricts the users in their creativity and ability to make detailed adjustments, such as brightness or contrast. We argue that incorporating user preferences by guiding an automated enhancement method simplifies image editing and increases the enhancement's focus on the user. This work thus proposes the Neural Image Correction & Enhancement Routine (NICER), a neural network based approach to no-reference image enhancement in a fully-, semi-automatic or fully manual process that is interactive and user-centered. NICER iteratively adjusts image editing parameters in order to maximize an aesthetic score based on image style and content. Users can modify these parameters at any time and guide the optimization process towards a desired direction. This interactive workflow is a novelty in the field of human-computer interaction for image enhancement tasks. In a user study, we show that NICER can improve image aesthetics without user interaction and that allowing user interaction leads to diverse enhancement outcomes that are strongly preferred over the unedited image. We make our code publicly available to facilitate further research in this direction.
SB-MTL: Score-based Meta Transfer-Learning for Cross-Domain Few-Shot Learning
Venue: arXiv cs.CV 2020
Year: 2020

While many deep learning methods have seen significant success in tackling the problem of domain adaptation and few-shot learning separately, far fewer methods are able to jointly tackle both problems in Cross-Domain Few-Shot Learning (CD-FSL). This problem is exacerbated under sharp domain shifts that typify common computer vision applications. In this paper, we present a novel, flexible and effective method to address the CD-FSL problem. Our method, called Score-based Meta Transfer-Learning (SB-MTL), combines transfer-learning and meta-learning by using a MAML-optimized feature encoder and a score-based Graph Neural Network. First, we have a feature encoder with specific layers designed to be fine-tuned. To do so, we apply a first-order MAML algorithm to find good initializations. Second, instead of directly taking the classification scores after fine-tuning, we interpret the scores as coordinates by mapping the pre-softmax classification scores onto a metric space. Subsequently, we apply a Graph Neural Network to propagate label information from the support set to the query set in our score-based metric space. We test our model on the Broader Study of Cross-Domain Few-Shot Learning (BSCD-FSL) benchmark, which includes a range of target domains with highly varying dissimilarity to the miniImagenet source domain. We observe significant improvements in accuracy across 5, 20 and 50 shot, and on the four target domains. In terms of average accuracy, our model outperforms previous transfer-learning methods by 5.93% and previous meta-learning methods by 14.28%.
We Dare You: A Lifecycle Study of a Substitutional Reality Installation in a Museum Space
Venue: arXiv cs.HC 2020
Year: 2020

In this article, we present a lifecycle study of We Dare You, a Substitutional Reality (SR) installation that combines visual and tactile stimuli. The installation is set up in a center for architecture, and invites visitors to explore its facade while playing with vertigo, in a visual Virtual Reality (VR) environment that replicates the surrounding physical space of the installation. Drawing on an ethnographic approach, including observations and interviews, we researched the exhibit from its opening, through the initial months plagued by technical problems, its subsequent success as a social and playful installation, on to its closure, due to COVID-19, and its subsequent reopening. Our findings explore the challenges caused by both the hybrid nature of the installation, as well as the visitor' playful use of the installation which made the experience social and performative - but also caused some problems. We also discuss the problems We Dare You faced in light of hygiene demands due to COVID-19. The analysis contrasts the design processes and expectations of stakeholders with the audience's playful appropriation, which led the stakeholders to see the installation as both a success and a failure. Evaluating the design and redesign through use on behalf of visitors, we argue that an approach that further opens up the post-production experience to a process of continuous redesign based on the user input - what has been termed "design-after-design" - could facilitate the design of similar experiences in the museum and heritage sector, supporting a participatory agenda in the design process, and helping to resolve the tension between stakeholders' expectations and visitors' playful appropriations.
PRESENT: An Android-Based Class Attendance Monitoring System Using Face Recognition Technology
Venue: arXiv cs.CY 2020
Year: 2020

The study aimed to develop an Android-Based Class Attendance Monitoring Application using Face Recognition to make attendance checking and monitoring easier and faster. The researcher used incremental model as the software development process and the application was evaluated by seventeen (17) faculty members .A validated evaluation questionnaire was used to rate the level of acceptability of the application based on ISO 9126 software quality and the level of satisfaction for its major features. For the statistical treatment of the data collected, Likert Scale, weighted mean and t-test were utilized by the researcher. The results revealed that instructors find the existing way of checking attendance as time consuming and a tedious task. Furthermore, the respondents assessed the developed application as moderately acceptable in terms of functionality, reliability and usability while portability was rated as highly acceptable. With regards to the features, the respondents were very satisfied. The researcher concluded that the developed application was useful and it can support the needs of the instructors to make attendance checking and monitoring easier, faster, and reliable. Due to its acceptable evaluation result, instructors should consider the use of this tool as an alternative to the existing process of checking and monitoring class attendance. With the integration of different technologies such as Android, face recognition and SMS, the traditional way of checking class attendance can be made easier, faster, reliable and secured, thus improving classroom management.
Transfer Learning as an Enabler of the Intelligent Digital Twin
Venue: arXiv cs.LG 2020
Year: 2020

Digital Twins have been described as beneficial in many areas, such as virtual commissioning, fault prediction or reconfiguration planning. Equipping Digital Twins with artificial intelligence functionalities can greatly expand those beneficial applications or open up altogether new areas of application, among them cross-phase industrial transfer learning. In the context of machine learning, transfer learning represents a set of approaches that enhance learning new tasks based upon previously acquired knowledge. Here, knowledge is transferred from one lifecycle phase to another in order to reduce the amount of data or time needed to train a machine learning algorithm. Looking at common challenges in developing and deploying industrial machinery with deep learning functionalities, embracing this concept would offer several advantages: Using an intelligent Digital Twin, learning algorithms can be designed, configured and tested in the design phase before the physical system exists and real data can be collected. Once real data becomes available, the algorithms must merely be fine-tuned, significantly speeding up commissioning and reducing the probability of costly modifications. Furthermore, using the Digital Twin's simulation capabilities virtually injecting rare faults in order to train an algorithm's response or using reinforcement learning, e.g. to teach a robot, become practically feasible. This article presents several cross-phase industrial transfer learning use cases utilizing intelligent Digital Twins. A real cyber physical production system consisting of an automated welding machine and an automated guided vehicle equipped with a robot arm is used to illustrate the respective benefits.
Effect of backdoor attacks over the complexity of the latent space distribution
Venue: arXiv cs.LG 2020
Year: 2020

The input space complexity determines the model's capabilities to extract their knowledge and translate the space of attributes into a function which is assumed in general, as a concatenation of non-linear functions between layers. In the presence of backdoor attacks, the space complexity changes, and induces similarities between classes that directly affect the model's training. As a consequence, the model tends to overfit the input set. In this research, we suggest the D-vine Copula Auto-Encoder (VCAE) as a tool to estimate the latent space distribution under the presence of backdoor triggers. Since no assumptions are made on the distribution estimation, like in Variational Autoencoders (VAE). It is possible to observe the backdoor stamp in non-attacked categories randomly generated. We exhibit the differences between a clean model (baseline) and the attacked one (backdoor) in a pairwise representation of the distribution. The idea is to illustrate the dependency structure change in the input space induced by backdoor features. Finally, we quantify the entropy's changes and the Kullback-Leibler divergence between models. In our results, we found the entropy in the latent space increases by around 27\% due to the backdoor trigger added to the input
Quasi-Newton's method in the class gradient defined high-curvature subspace
Venue: arXiv cs.LG 2020
Year: 2020

Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes. Moreover, this subspace corresponds to the subspace spanned by the logit gradients for each class. An obvious strategy to speed up optimisation would be to use Newton's method in the high-curvature subspace and stochastic gradient descent in the co-space. We show that a naive implementation actually slows down convergence and we speculate why this might be.
Budget Constrained Interactive Search for Multiple Targets
Venue: arXiv cs.DB 2020
Year: 2020

Interactive graph search leverages human intelligence to categorize target labels in a hierarchy, which are useful for image classification, product categorization, and database search. However, many existing studies of interactive graph search aim at identifying a single target optimally, and suffer from the limitations of asking too many questions and not being able to handle multiple targets. To address these two limitations, in this paper, we study a new problem of budget constrained interactive graph search for multiple targets called kBM-IGS-problem. Specifically, given a set of multiple targets T in a hierarchy, and two parameters k and b, the goal is to identify a k-sized set of selections S such that the closeness between selections S and targets T is as small as possible, by asking at most a budget of b questions. We theoretically analyze the updating rules and design a penalty function to capture the closeness between selections and targets. To tackle the kBM-IGS-problem, we develop a novel framework to ask questions using the best vertex with the largest expected gain, which makes a balanced trade-off between target probability and benefit gain. Based on the kBM-IGS framework, we first propose an efficient algorithm STBIS to handle the SingleTarget problem, which is a special case of kBM-IGS. Then, we propose a dynamic programming based method kBM-DP to tackle the MultipleTargets problem. To further improve efficiency, we propose two heuristic but efficient algorithms kBM-Topk and kBM-DP+. kBM-Topk develops a variant gain function and selects the top-k vertices independently. kBM-DP+ uses an upper bound of gains and prunes disqualified vertices to save computations. Experiments on large real-world datasets with ground-truth targets verify both the effectiveness and efficiency of our proposed algorithms.
IMAGO: A family photo album dataset for a socio-historical analysis of the twentieth century
Venue: arXiv cs.CV 2020
Year: 2020

Although one of the most popular practices in photography since the end of the 19th century, an increase in scholarly interest in family photo albums dates back to the early 1980s. Such collections of photos may reveal sociological and historical insights regarding specific cultures and times. They are, however, in most cases scattered among private homes and only available on paper or photographic film, thus making their analysis by academics such as historians, social-cultural anthropologists and cultural theorists very cumbersome. In this paper, we analyze the IMAGO dataset including photos belonging to family albums assembled at the University of Bologna's Rimini campus since 2004. Following a deep learning-based approach, the IMAGO dataset has offered the opportunity of experimenting with photos taken between year 1845 and year 2009, with the goals of assessing the dates and the socio-historical contexts of the images, without use of any other sources of information. Exceeding our initial expectations, such analysis has revealed its merit not only in terms of the performance of the approach adopted in this work, but also in terms of the foreseeable implications and use for the benefit of socio-historical research. To the best of our knowledge, this is the first work that moves along this path in literature.
Sketching Persistence Diagrams
Venue: arXiv cs.CG 2020
Year: 2020

Given a persistence diagram with $n$ points, we give an algorithm that produces a sequence of $n$ persistence diagrams converging in bottleneck distance to the input diagram, the $i$th of which has $i$ distinct (weighted) points and is a $2$-approximation to the closest persistence diagram with that many distinct points. For each approximation, we precompute the optimal matching between the $i$th and the $(i+1)$st. Perhaps surprisingly, the entire sequence of diagrams as well as the sequence of matchings can be represented in $O(n)$ space. The main approach is to use a variation of the greedy permutation of the persistence diagram to give good Hausdorff approximations and assign weights to these subsets. We give a new algorithm to efficiently compute this permutation, despite the high implicit dimension of points in a persistence diagram due to the effect of the diagonal. The sketches are also structured to permit fast (linear time) approximations to the Hausdorff distance between diagrams -- a lower bound on the bottleneck distance. For approximating the bottleneck distance, sketches can also be used to compute a linear-size neighborhood graph directly, obviating the need for geometric data structures used in state-of-the-art methods for bottleneck computation.
Artificial Intelligence Accelerators based on Graphene Optoelectronic Devices
Venue: arXiv cs.ET 2020
Year: 2020

Optical and optoelectronic approaches of performing matrix-vector multiplication (MVM) operations have shown the great promise of accelerating machine learning (ML) algorithms with unprecedented performance. The incorporation of nanomaterials into the system can further improve the performance thanks to their extraordinary properties, but the non-uniformity and variation of nanostructures in the macroscopic scale pose severe limitations for large-scale hardware deployment. Here, we report a new optoelectronic architecture consisting of spatial light modulators and photodetector arrays made from graphene to perform MVM. The ultrahigh carrier mobility of graphene, nearly-zero-power-consumption electro-optic control, and extreme parallelism suggest ultrahigh data throughput and ultralow-power consumption. Moreover, we develop a methodology of performing accurate calculations with imperfect components, laying the foundation for scalable systems. Finally, we perform a few representative ML algorithms, including singular value decomposition, support vector machine, and deep neural networks, to show the versatility and generality of our platform.
Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we investigate a radio access network (RAN) slicing problem for Internet of vehicles (IoV) services with different quality of service (QoS) requirements, in which multiple logically-isolated slices are constructed on a common roadside network infrastructure. A dynamic RAN slicing framework is presented to dynamically allocate radio spectrum and computing resource, and distribute computation workloads for the slices. To obtain an optimal RAN slicing policy for accommodating the spatial-temporal dynamics of vehicle traffic density, we first formulate a constrained RAN slicing problem with the objective to minimize long-term system cost. This problem cannot be directly solved by traditional reinforcement learning (RL) algorithms due to complicated coupled constraints among decisions. Therefore, we decouple the problem into a resource allocation subproblem and a workload distribution subproblem, and propose a two-layer constrained RL algorithm, named Resource Allocation and Workload diStribution (RAWS) to solve them. Specifically, an outer layer first makes the resource allocation decision via an RL algorithm, and then an inner layer makes the workload distribution decision via an optimization subroutine. Extensive trace-driven simulations show that the RAWS effectively reduces the system cost while satisfying QoS requirements with a high probability, as compared with benchmarks.
Aerial Imagery Pixel-level Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Aerial imagery can be used for important work on a global scale. Nevertheless, the analysis of this data using neural network architectures lags behind the current state-of-the-art on popular datasets such as PASCAL VOC, CityScapes and Camvid. In this paper we bridge the performance-gap between these popular datasets and aerial imagery data. Little work is done on aerial imagery with state-of-the-art neural network architectures in a multi-class setting. Our experiments concerning data augmentation, normalisation, image size and loss functions give insight into a high performance setup for aerial imagery segmentation datasets. Our work, using the state-of-the-art DeepLabv3+ Xception65 architecture, achieves a mean IOU of 70% on the DroneDeploy validation set. With this result, we clearly outperform the current publicly available state-of-the-art validation set mIOU (65%) performance with 5%. Furthermore, to our knowledge, there is no mIOU benchmark for the test set. Hence, we also propose a new benchmark on the DroneDeploy test set using the best performing DeepLabv3+ Xception65 architecture, with a mIOU score of 52.5%.