OpenRaiser/Intern-Atlas huggingface dataset:
BiOpt: Bi-Level Optimization for Few-Shot Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

Few-shot segmentation is a challenging task that aims to segment objects of new classes given scarce support images. In the inductive setting, existing prototype-based methods focus on extracting prototypes from the support images; however, they fail to utilize semantic information of the query images. In this paper, we propose Bi-level Optimization (BiOpt), which succeeds to compute class prototypes from the query images under inductive setting. The learning procedure of BiOpt is decomposed into two nested loops: inner and outer loop. On each task, the inner loop aims to learn optimized prototypes from the query images. An init step is conducted to fully exploit knowledge from both support and query features, so as to give reasonable initialized prototypes into the inner loop. The outer loop aims to learn a discriminative embedding space across different tasks. Extensive experiments on two benchmarks verify the superiority of our proposed BiOpt algorithm. In particular, we consistently achieve the state-of-the-art performance on 5-shot PASCAL-$5^i$ and 1-shot COCO-$20^i$.
RVCoreP-32IC: A high-performance RISC-V soft processor with an efficient fetch unit supporting the compressed instructions
Venue: arXiv cs.AR 2020
Year: 2020

In this paper, we propose a high-performance RISC-V soft processor with an efficient fetch unit supporting the compressed instructions targeting on FPGA. The compressed instruction extension in RISC-V can reduce the program size by about 25%. But it needs a complicated logic for the instruction fetch unit and has a significant impact on performance. We propose an instruction fetch unit that supports the compressed instructions while exhibiting high performance. Furthermore, we propose a RISC-V soft processor using this unit. We implement this proposed processor in Verilog HDL and verify the behavior using Verilog simulation and an actual Xilinx Atrix-7 FPGA board. We compare the results of some benchmarks and the amount of hardware with related works. DMIPS, CoreMark value, and Embench value of the proposed processor achieved 42.5%, 41.1% and 21.3% higher performance than the related work, respectively.
Application of Facial Recognition using Convolutional Neural Networks for Entry Access Control
Venue: arXiv cs.CV 2020
Year: 2020

The purpose of this paper is to design a solution to the problem of facial recognition by use of convolutional neural networks, with the intention of applying the solution in a camera-based home-entry access control system. More specifically, the paper focuses on solving the supervised classification problem of taking images of people as input and classifying the person in the image as one of the authors or not. Two approaches are proposed: (1) building and training a neural network called WoodNet from scratch and (2) leveraging transfer learning by utilizing a network pre-trained on the ImageNet database and adapting it to this project's data and classes. In order to train the models to recognize the authors, a dataset containing more than 150 000 images has been created, balanced over the authors and others. Image extraction from videos and image augmentation techniques were instrumental for dataset creation. The results are two models classifying the individuals in the dataset with high accuracy, achieving over 99% accuracy on held-out test data. The pre-trained model fitted significantly faster than WoodNet, and seems to generalize better. However, these results come with a few caveats. Because of the way the dataset was compiled, as well as the high accuracy, one has reason to believe the models over-fitted to the data to some degree. An added consequence of the data compilation method is that the test dataset may not be sufficiently different from the training data, limiting its ability to validate generalization of the models. However, utilizing the models in a web-cam based system, classifying faces in real-time, shows promising results and indicates that the models generalized fairly well for at least some of the classes (see the accompanying video).
Evaluating Input Representation for Language Identification in Hindi-English Code Mixed Text
Venue: arXiv cs.CL 2020
Year: 2020

Natural language processing (NLP) techniques have become mainstream in the recent decade. Most of these advances are attributed to the processing of a single language. More recently, with the extensive growth of social media platforms focus has shifted to code-mixed text. The code-mixed text comprises text written in more than one language. People naturally tend to combine local language with global languages like English. To process such texts, current NLP techniques are not sufficient. As a first step, the text is processed to identify the language of the words in the text. In this work, we focus on language identification in code-mixed sentences for Hindi-English mixed text. The task of language identification is formulated as a token classification task. In the supervised setting, each word in the sentence has an associated language label. We evaluate different deep learning models and input representation combinations for this task. Mainly, character, sub-word, and word embeddings are considered in combination with CNN and LSTM based models. We show that sub-word representation along with the LSTM model gives the best results. In general sub-word representations perform significantly better than other input representations. We report the best accuracy of 94.52% using a single layer LSTM model on the standard SAIL ICON 2017 test set.
Evolutionary Planning in Latent Space
Venue: arXiv cs.LG 2020
Year: 2020

Planning is a powerful approach to reinforcement learning with several desirable properties. However, it requires a model of the world, which is not readily available in many real-life problems. In this paper, we propose to learn a world model that enables Evolutionary Planning in Latent Space (EPLS). We use a Variational Auto Encoder (VAE) to learn a compressed latent representation of individual observations and extend a Mixture Density Recurrent Neural Network (MDRNN) to learn a stochastic, multi-modal forward model of the world that can be used for planning. We use the Random Mutation Hill Climbing (RMHC) to find a sequence of actions that maximize expected reward in this learned model of the world. We demonstrate how to build a model of the world by bootstrapping it with rollouts from a random policy and iteratively refining it with rollouts from an increasingly accurate planning policy using the learned world model. After a few iterations of this refinement, our planning agents are better than standard model-free reinforcement learning approaches demonstrating the viability of our approach.
Legacy Photo Editing with Learned Noise Prior
Venue: arXiv cs.CV 2020
Year: 2020

There are quite a number of photographs captured under undesirable conditions in the last century. Thus, they are often noisy, regionally incomplete, and grayscale formatted. Conventional approaches mainly focus on one point so that those restoration results are not perceptually sharp or clean enough. To solve these problems, we propose a noise prior learner NEGAN to simulate the noise distribution of real legacy photos using unpaired images. It mainly focuses on matching high-frequency parts of noisy images through discrete wavelet transform (DWT) since they include most of noise statistics. We also create a large legacy photo dataset for learning noise prior. Using learned noise prior, we can easily build valid training pairs by degrading clean images. Then, we propose an IEGAN framework performing image editing including joint denoising, inpainting and colorization based on the estimated noise prior. We evaluate the proposed system and compare it with state-of-the-art image enhancement methods. The experimental results demonstrate that it achieves the best perceptual quality. https://github.com/zhaoyuzhi/Legacy-Photo-Editing-with-Learned-Noise-Prior for the codes and the proposed LP dataset.
Corona-Warn-App: Erste Ergebnisse einer Onlineumfrage zur (Nicht-)Nutzung und Gebrauch
Venue: arXiv cs.HC 2020
Year: 2020

In this study, the German "Corona-Warn-App" of the German Federal Government and the Robert-Koch-Institute is examined by means of a non-representative online survey with 1482 participants for reasons of use and non-use. The study provides insights into user behavior with the app during the Corona pandemic, highlights the topic of data protection and how the app is used in general. Our results show that the app is often not used due to privacy concerns, but that there are also technical problems and doubts about its usefulness. In addition, the app is mainly used due to altruistic reasons and is often opened to view the own risk assessment and to ensure its functionality. To better understand the results, we compare our results with a sample of infas 360 with 10553 participants. It is shown that the results of this study can be compared to a larger population. Finally, the results are discussed and recommendations for action are derived.
The Diary of Niels: Affective engagement through tangible interaction with museum artifacts
Venue: arXiv cs.HC 2020
Year: 2020

This paper presents a research through design exploration using tangible interactions in order to seamlessly integrate technology in a historical house museum. The study addresses a longstanding concern in museum exhibition design that interactive technologies may distract from the artifacts on display. Through an iterative design process including user studies, a co-creation workshop with museum staff and several prototypes, we developed an interactive installation called The Diary of Niels that combines physical objects, RFID sensors and an elaborate fiction in order to facilitate increased visitor engagement. Insights from the research process and user tests indicate that the integration of technology and artifacts is meaningful and engaging for users, and helps introduce museum visitors to the historic theme of the exhibition and the meaning of the artifacts. The study also points to continued challenges in integrating such hybrid experiences fully with the rest of the exhibition.
Designing for Interpersonal Museum Experiences
Venue: arXiv cs.HC 2020
Year: 2020

What does the age of participation look like from the perspective of a museum visitor? Arguably, the concept of participative experiences is already so deeply ingrained in our culture that we may not even think about it as participation. Museum visitors engage in a number of activities, of which observing the exhibits is only one part. Since most visitors come to the museum together with someone else, they spend time and attention on the people they came with, and often the needs of the group are given priority over individual preferences. How can museums tap into these activities - and make themselves relevant to visitors? In this chapter we will try to approach this constructively, as a design opportunity. Could it be productive for the museum to consider itself not only as a disseminator of knowledge, but also as the facilitator of participative activities between visitors? In what follows, we will outline a range of practical design projects that serve as examples of this approach. These projects were part of the European Union funded Horizon2020 project GIFT, a cross-disciplinary collaboration between researchers, artists, designers and many international museums and heritage organisations, exploring the concept of interpersonal museum experiences (see https://gifting.digital/). What the projects have in common is that they build on visitors co-creating and sharing their own narratives in the museum context. We suggest that these projects demonstrate a spectrum of possibilities: From experiences that take place almost without any museum involvement, to those that give museums a role in curating these narratives.
Multi-task Learning for Human Settlement Extent Regression and Local Climate Zone Classification
Venue: arXiv cs.CV 2020
Year: 2020

Human Settlement Extent (HSE) and Local Climate Zone (LCZ) maps are both essential sources, e.g., for sustainable urban development and Urban Heat Island (UHI) studies. Remote sensing (RS)- and deep learning (DL)-based classification approaches play a significant role by providing the potential for global mapping. However, most of the efforts only focus on one of the two schemes, usually on a specific scale. This leads to unnecessary redundancies, since the learned features could be leveraged for both of these related tasks. In this letter, the concept of multi-task learning (MTL) is introduced to HSE regression and LCZ classification for the first time. We propose a MTL framework and develop an end-to-end Convolutional Neural Network (CNN), which consists of a backbone network for shared feature learning, attention modules for task-specific feature learning, and a weighting strategy for balancing the two tasks. We additionally propose to exploit HSE predictions as a prior for LCZ classification to enhance the accuracy. The MTL approach was extensively tested with Sentinel-2 data of 13 cities across the world. The results demonstrate that the framework is able to provide a competitive solution for both tasks.
Generative Adversarial Simulator
Venue: arXiv cs.LG 2020
Year: 2020

Knowledge distillation between machine learning models has opened many new avenues for parameter count reduction, performance improvements, or amortizing training time when changing architectures between the teacher and student network. In the case of reinforcement learning, this technique has also been applied to distill teacher policies to students. Until now, policy distillation required access to a simulator or real world trajectories. In this paper we introduce a simulator-free approach to knowledge distillation in the context of reinforcement learning. A key challenge is having the student learn the multiplicity of cases that correspond to a given action. While prior work has shown that data-free knowledge distillation is possible with supervised learning models by generating synthetic examples, these approaches to are vulnerable to only producing a single prototype example for each class. We propose an extension to explicitly handle multiple observations per output class that seeks to find as many exemplars as possible for a given output class by reinitializing our data generator and making use of an adversarial loss. To the best of our knowledge, this is the first demonstration of simulator-free knowledge distillation between a teacher and a student policy. This new approach improves over the state of the art on data-free learning of student networks on benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10), and we also demonstrate that it specifically tackles issues with multiple input modes. We also identify open problems when distilling agents trained in high dimensional environments such as Pong, Breakout, or Seaquest.
Consolidation via Policy Information Regularization in Deep RL for Multi-Agent Games
Venue: arXiv cs.AI 2020
Year: 2020

This paper introduces an information-theoretic constraint on learned policy complexity in the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) reinforcement learning algorithm. Previous research with a related approach in continuous control experiments suggests that this method favors learning policies that are more robust to changing environment dynamics. The multi-agent game setting naturally requires this type of robustness, as other agents' policies change throughout learning, introducing a nonstationary environment. For this reason, recent methods in continual learning are compared to our approach, termed Capacity-Limited MADDPG. Results from experimentation in multi-agent cooperative and competitive tasks demonstrate that the capacity-limited approach is a good candidate for improving learning performance in these environments.
An Interactive Foreign Language Trainer Using Assessment and Feedback Modalities
Venue: arXiv cs.CY 2020
Year: 2020

English has long been set as the universal language. Basically most, if not all countries in the world know how to speak English or at least try to use it in their everyday communications for the purpose of globalizing. This study is designed to help the students learn from one or all of the four most commonly used foreign languages in the field of Information Technology namely Korean, Mandarin Chinese, Japanese, and Spanish. Composed of a set of words, phrases, and sentences, the program is intended to quickly teach the students in the form of basic, intermediate, and advanced levels. This study has used the Agile model in system development. Functionality, reliability, usability, efficiency, and portability were also considered in determining the level of the acceptability of the system in terms of ISO 25010:2011. This interactive foreign language trainer is built to associate fun with learning, to remedy the lack of perseverance by some in learning a new language, and to make learning the users' favorite playtime activity. The study allows the user to interact with the program which provides support for their learning. Moreover, this study reveals that integrating feedback modalities in the training and assessment modules of the software strengthens and enhances the memory in learning the language.
Elastic Interaction of Particles for Robotic Tactile Simulation
Venue: arXiv cs.RO 2020
Year: 2020

Tactile sensing plays an important role in robotic perception and manipulation. To overcome the real-world limitations of data collection, simulating tactile response in virtual environment comes as a desire direction of robotic research. Most existing works model the tactile sensor as a rigid multi-body, which is incapable of reflecting the elastic property of the tactile sensor as well as characterizing the fine-grained physical interaction between two objects. In this paper, we propose Elastic Interaction of Particles (EIP), a novel framework for tactile emulation. At its core, EIP models the tactile sensor as a group of coordinated particles, and the elastic theory is applied to regulate the deformation of particles during the contact process. The implementation of EIP is conducted from scratch, without resorting to any existing physics engine. Experiments to verify the effectiveness of our method have been carried out on two applications: robotic perception with tactile data and 3D geometric reconstruction by tactile-visual fusion. It is possible to open up a new vein for robotic tactile simulation, and contribute to various downstream robotic tasks.
Exploring Alternatives to Softmax Function
Venue: arXiv cs.LG 2020
Year: 2020

Softmax function is widely used in artificial neural networks for multiclass classification, multilabel classification, attention mechanisms, etc. However, its efficacy is often questioned in literature. The log-softmax loss has been shown to belong to a more generic class of loss functions, called spherical family, and its member log-Taylor softmax loss is arguably the best alternative in this class. In another approach which tries to enhance the discriminative nature of the softmax function, soft-margin softmax (SM-softmax) has been proposed to be the most suitable alternative. In this work, we investigate Taylor softmax, SM-softmax and our proposed SM-Taylor softmax, an amalgamation of the earlier two functions, as alternatives to softmax function. Furthermore, we explore the effect of expanding Taylor softmax up to ten terms (original work proposed expanding only to two terms) along with the ramifications of considering Taylor softmax to be a finite or infinite series during backpropagation. Our experiments for the image classification task on different datasets reveal that there is always a configuration of the SM-Taylor softmax function that outperforms the normal softmax function and its other alternatives.
Conformance Checking of Mixed-paradigm Process Models
Venue: arXiv cs.FL 2020
Year: 2020

Mixed-paradigm process models integrate strengths of procedural and declarative representations like Petri nets and Declare. They are specifically interesting for process mining because they allow capturing complex behaviour in a compact way. A key research challenge for the proliferation of mixed-paradigm models for process mining is the lack of corresponding conformance checking techniques. In this paper, we address this problem by devising the first approach that works with intertwined state spaces of mixed-paradigm models. More specifically, our approach uses an alignment-based replay to explore the state space and compute trace fitness in a procedural way. In every state, the declarative constraints are separately updated, such that violations disable the corresponding activities. Our technique provides for an efficient replay towards an optimal alignment by respecting all orthogonal Declare constraints. We have implemented our technique in ProM and demonstrate its performance in an evaluation with real-world event logs.
FERN: Fair Team Formation for Mutually Beneficial Collaborative Learning
Venue: arXiv cs.CY 2020
Year: 2020

Automated Team Formation is becoming increasingly important for a plethora of applications in open source community projects, remote working platforms, as well as online educational systems. The latter case, in particular, poses significant challenges that are specific to the educational domain. Indeed, teaming students aims to accomplish far more than the successful completion of a specific task. It needs to ensure that all members in the team benefit from the collaborative work, while also ensuring that the participants are not discriminated with respect to their protected attributes, such as race and gender. Towards achieving these goals, this work introduces FERN, a fair team formation approach that promotes mutually beneficial peer learning, dictated by protected group fairness as equality of opportunity in collaborative learning. We formulate the problem as a multi-objective discrete optimization problem. We show this problem to be NP-hard and propose a heuristic hill-climbing algorithm. Extensive experiments on both synthetic and real-world datasets against well-known team formation techniques show the effectiveness of the proposed method.
Cycle-consistent Generative Adversarial Networks for Neural Style Transfer using data from Chang'E-4
Venue: arXiv cs.CV 2020
Year: 2020

Generative Adversarial Networks (GANs) have had tremendous applications in Computer Vision. Yet, in the context of space science and planetary exploration the door is open for major advances. We introduce tools to handle planetary data from the mission Chang'E-4 and present a framework for Neural Style Transfer using Cycle-consistency from rendered images. The experiments are conducted in the context of the Iris Lunar Rover, a nano-rover that will be deployed in lunar terrain in 2021 as the flagship of Carnegie Mellon, being the first unmanned rover of America to be on the Moon.