OpenRaiser/Intern-Atlas huggingface dataset:
Connected Learning, Collapsed Contexts
Venue: arXiv cs.HC 2021
Year: 2021

Researchers and designers have incorporated social media affordances into learning technologies to engage young people and support personally relevant learning, but youth may reject these attempts because they do not meet user expectations. Through in-depth case studies, we explore the sociotechnical ecosystems of six teens (ages 15-18) working at a science center that had recently introduced a digital badge system to track and recognize their learning. By analyzing interviews, observations, ecological momentary assessments, and system data, we examined tensions in how badges as connected learning technologies operate in teens' sociotechnical ecosystems. We found that, due to issues of unwanted context collapse and incongruent identity representations, youth only used certain affordances of the system and did so sporadically. Additionally, we noted that some features seemed to prioritize values of adult stakeholders over youth. Using badges as a lens, we reveal critical tensions and offer design recommendations for networked learning technologies.
Coded Compressed Sensing with Successive Cancellation List Decoding for Unsourced Random Access with Massive MIMO
Venue: arXiv cs.IT 2021
Year: 2021

Unsourced random access (URA) is an increasingly popular communication paradigm attuned to machine driven data transfers in \textit{Internet-of-Things} (IoT) networks. In a typical URA setting, a small subset of active devices within a very large population wish to transmit short messages to a central base station. Originally defined for base stations equipped with a single antenna, the URA paradigm has recently been extended to practical scenarios involving base stations with a massive number of antennas by Fengler et al. The proposed concatenated coding scheme therein utilizes a non-Bayesian sparse recovery algorithm coupled with the tree code introduced by Amalladinne et al. in the context of coded compressed sensing. Currently, the existing MIMO implementation uses these two components in isolation. This article introduces an enhanced successive cancellation list decoding style scheme that facilitates dynamic interactions between the sparse recovery algorithm and the tree decoder. This modification can reduce the search space of the AD algorithm drastically; and it results in significant improvements both in terms of error performance and computational complexity. Simulation results reveal that, for a system with 100 active users, the proposed decoder reduces the number of required antennas at the base station by 23\% to achieve a performance akin to the scheme by Fengler et al.
The Kids Are / Not / Sort of All Right
Venue: arXiv cs.HC 2021
Year: 2021

We investigated changes in and factors affecting American adolescents' subjective wellbeing during the early months (April - August 2020) of the coronavirus pandemic in the United States. Twenty-one teens (14 - 19 years) participated in interviews at the start and end of the study and completed ecological momentary assessments three times per week between the interviews. There was an aggregate trend toward increased wellbeing, with considerable variation within and across participants. Teens reported greater reliance on networked technologies as their unstructured time increased during lockdown. Using multilevel growth modeling, we found that how much total time teens spent with technology had less bearing on daily fluctuations in wellbeing than the satisfaction and meaning they derived from their technology use. Ultimately, teens felt online communication could not replace face-to-face interactions. We conducted two follow-up participatory design sessions with nine teens to explore these insights in greater depth and reflect on general implications for design to support teens' meaningful technology experiences and wellbeing during disruptive life events.
A Generalized Concurrent Rule Construction for Double-Pushout Rewriting
Venue: arXiv cs.LO 2021
Year: 2021

Double-pushout rewriting is an established categorical approach to the rule-based transformation of graphs and graph-like objects. One of its standard results is the construction of concurrent rules and the Concurrency Theorem pertaining to it: The sequential application of two rules can equivalently be replaced by the application of a concurrent rule and vice versa. We extend and generalize this result by introducing generalized concurrent rules (GCRs). Their distinguishing property is that they allow identifying and preserving elements that are deleted by their first underlying rule and created by the second one. We position this new kind of composition of rules among the existing ones and obtain a Generalized Concurrency Theorem for it. We conduct our work in the same generic framework in which the Concurrency Theorem has been presented, namely double-pushout rewriting in M-adhesive categories via rules equipped with application conditions.
iCub
Venue: arXiv cs.RO 2021
Year: 2021

In this chapter we describe the history and evolution of the iCub humanoid platform. We start by describing the first version as it was designed during the RobotCub EU project and illustrate how it evolved to become the platform that is adopted by more than 30 laboratories world wide. We complete the chapter by illustrating some of the research activities that are currently carried out on the iCub robot, i.e. visual perception, event driven sensing and dynamic control. We conclude the Chapter with a discussion of the lessons we learned and a preview of the upcoming next release of the robot, iCub 3.0.
DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Data
Venue: arXiv cs.CV 2021
Year: 2021

Despite over two decades of progress, imbalanced data is still considered a significant challenge for contemporary machine learning models. Modern advances in deep learning have magnified the importance of the imbalanced data problem. The two main approaches to address this issue are based on loss function modifications and instance resampling. Instance sampling is typically based on Generative Adversarial Networks (GANs), which may suffer from mode collapse. Therefore, there is a need for an oversampling method that is specifically tailored to deep learning models, can work on raw images while preserving their properties, and is capable of generating high quality, artificial images that can enhance minority classes and balance the training set. We propose DeepSMOTE - a novel oversampling algorithm for deep learning models. It is simple, yet effective in its design. It consists of three major components: (i) an encoder/decoder framework; (ii) SMOTE-based oversampling; and (iii) a dedicated loss function that is enhanced with a penalty term. An important advantage of DeepSMOTE over GAN-based oversampling is that DeepSMOTE does not require a discriminator, and it generates high-quality artificial images that are both information-rich and suitable for visual inspection. DeepSMOTE code is publicly available at: https://github.com/dd1github/DeepSMOTE
Migrating Client Code without Change Examples
Venue: arXiv cs.SE 2021
Year: 2021

API developers evolve software libraries to fix bugs, add new features, or refactor code. To benefit from such library evolution, the programmers of client projects have to repetitively upgrade their library usages and adapt their codebases to any library API breaking changes (e.g., API renaming). Such adaptive changes can be tedious and error-prone. Existing tools provide limited support to help programmers migrate client projects from old library versions to new ones. For instance, some tools extract API mappings be-tween library versions and only suggest simple adaptive changes (i.e., statement updates); other tools suggest or automate more complicated edits (e.g., statement insertions) based on user-provided exemplar code migrations. However, when new library versions are available, it is usually cumbersome and time-consuming for users to provide sufficient human-crafted samples in order to guide automatic migration. In this paper, we propose a novel approach, AutoUpdate, to further improve the state of the art. Instead of learning from change examples, we designed AutoUpdate to automate migration in a compiler-directed way. Namely, given a compilation error triggered by upgrading libraries, AutoUpdate exploits 13 migration opera-tors to generate candidate edits, and tentatively applies each edit until the error is resolved or all edits are explored. We conducted two experiments. The first experiment involves migrating 371 tutorial examples between versions of 5 popular libraries. AutoUpdate reduced migration-related compilation errors for 92.7% of tasks. It eliminated such errors for 32.4% of tasks, and 33.9% of the tasks have identical edits to manual migrations. In the second experiment, we applied AutoUpdate to migrate two real client projects of lucene. AutoUpdate successfully migrated both projects, and the migrated code passed all tests.
In the Danger Zone: U-Net Driven Quantile Regression can Predict High-risk SARS-CoV-2 Regions via Pollutant Particulate Matter and Satellite Imagery
Venue: arXiv cs.CV 2021
Year: 2021

Since the outbreak of COVID-19 policy makers have been relying upon non-pharmacological interventions to control the outbreak. With air pollution as a potential transmission vector there is need to include it in intervention strategies. We propose a U-net driven quantile regression model to predict $PM_{2.5}$ air pollution based on easily obtainable satellite imagery. We demonstrate that our approach can reconstruct $PM_{2.5}$ concentrations on ground-truth data and predict reasonable $PM_{2.5}$ values with their spatial distribution, even for locations where pollution data is unavailable. Such predictions of $PM_{2.5}$ characteristics could crucially advise public policy strategies geared to reduce the transmission of and lethality of COVID-19.
Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Encoder-decoder models have made great progress on handwritten mathematical expression recognition recently. However, it is still a challenge for existing methods to assign attention to image features accurately. Moreover, those encoder-decoder models usually adopt RNN-based models in their decoder part, which makes them inefficient in processing long $\LaTeX{}$ sequences. In this paper, a transformer-based decoder is employed to replace RNN-based ones, which makes the whole model architecture very concise. Furthermore, a novel training strategy is introduced to fully exploit the potential of the transformer in bidirectional language modeling. Compared to several methods that do not use data augmentation, experiments demonstrate that our model improves the ExpRate of current state-of-the-art methods on CROHME 2014 by 2.23%. Similarly, on CROHME 2016 and CROHME 2019, we improve the ExpRate by 1.92% and 2.28% respectively.
Development of a Fast and Robust Gaze Tracking System for Game Applications
Venue: arXiv cs.CV 2021
Year: 2021

In this study, a novel eye tracking system using a visual camera is developed to extract human's gaze, and it can be used in modern game machines to bring new and innovative interactive experience to players. Central to the components of the system, is a robust iris-center and eye-corner detection algorithm basing on it the gaze is continuously and adaptively extracted. Evaluation tests were applied to nine people to evaluate the accuracy of the system and the results were 2.50 degrees (view angle) in horizontal direction and 3.07 degrees in vertical direction.
Generalized Multimodal ELBO
Venue: arXiv cs.LG 2021
Year: 2021

Multiple data types naturally co-occur when describing real-world phenomena and learning from them is a long-standing goal in machine learning research. However, existing self-supervised generative models approximating an ELBO are not able to fulfill all desired requirements of multimodal models: their posterior approximation functions lead to a trade-off between the semantic coherence and the ability to learn the joint data distribution. We propose a new, generalized ELBO formulation for multimodal data that overcomes these limitations. The new objective encompasses two previous methods as special cases and combines their benefits without compromises. In extensive experiments, we demonstrate the advantage of the proposed method compared to state-of-the-art models in self-supervised, generative learning tasks.
Positional Injectivity for Innocent Strategies
Venue: arXiv cs.LO 2021
Year: 2021

In asynchronous games, Melli{è}s proved that innocent strategies are positional: their behaviour only depends on the position, not the temporal order used to reach it. This insightful result shaped our understanding of the link between dynamic (i.e. game) and static (i.e. relational) semantics. In this paper, we investigate the positionality of innocent strategies in the traditional setting of Hyland-Ong-Nickau-Coquand pointer games. We show that though innocent strategies are not positional, total finite innocent strategies still enjoy a key consequence of positionality, namely positional injectivity: they are entirely determined by their positions. Unfortunately, this does not hold in general: we show a counterexample if finiteness and totality are lifted. For finite partial strategies we leave the problem open; we show however the partial result that two strategies with the same positions must have the same P-views of maximal length.
Towards General Natural Language Understanding with Probabilistic Worldbuilding
Venue: arXiv cs.CL 2021
Year: 2021

We introduce the Probabilistic Worldbuilding Model (PWM), a new fully-symbolic Bayesian model of semantic parsing and reasoning, as a first step in a research program toward more domain- and task-general NLU and AI. Humans create internal mental models of their observations which greatly aid in their ability to understand and reason about a large variety of problems. In PWM, the meanings of sentences, acquired facts about the world, and intermediate steps in reasoning are all expressed in a human-readable formal language, with the design goal of interpretability. PWM is Bayesian, designed specifically to be able to generalize to new domains and new tasks. We derive and implement an inference algorithm that reads sentences by parsing and abducing updates to its latent world model that capture the semantics of those sentences, and evaluate it on two out-of-domain question-answering datasets: (1) ProofWriter and (2) a new dataset we call FictionalGeoQA, designed to be more representative of real language but still simple enough to focus on evaluating reasoning ability, while being robust against heuristics. Our method outperforms baselines on both, thereby demonstrating its value as a proof-of-concept.
Federated Face Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Face recognition has been extensively studied in computer vision and artificial intelligence communities in recent years. An important issue of face recognition is data privacy, which receives more and more public concerns. As a common privacy-preserving technique, Federated Learning is proposed to train a model cooperatively without sharing data between parties. However, as far as we know, it has not been successfully applied in face recognition. This paper proposes a framework named FedFace to innovate federated learning for face recognition. Specifically, FedFace relies on two major innovative algorithms, Partially Federated Momentum (PFM) and Federated Validation (FV). PFM locally applies an estimated equivalent global momentum to approximating the centralized momentum-SGD efficiently. FV repeatedly searches for better federated aggregating weightings via testing the aggregated models on some private validation datasets, which can improve the model's generalization ability. The ablation study and extensive experiments validate the effectiveness of the FedFace method and show that it is comparable to or even better than the centralized baseline in performance.
(ASNA) An Attention-based Siamese-Difference Neural Network with Surrogate Ranking Loss function for Perceptual Image Quality Assessment
Venue: arXiv cs.CV 2021
Year: 2021

Recently, deep convolutional neural networks (DCNN) that leverage the adversarial training framework for image restoration and enhancement have significantly improved the processed images' sharpness. Surprisingly, although these DCNNs produced crispier images than other methods visually, they may get a lower quality score when popular measures are employed for evaluating them. Therefore it is necessary to develop a quantitative metric to reflect their performances, which is well-aligned with the perceived quality of an image. Famous quantitative metrics such as Peak signal-to-noise ratio (PSNR), The structural similarity index measure (SSIM), and Perceptual Index (PI) are not well-correlated with the mean opinion score (MOS) for an image, especially for the neural networks trained with adversarial loss functions. This paper has proposed a convolutional neural network using an extension architecture of the traditional Siamese network so-called Siamese-Difference neural network. We have equipped this architecture with the spatial and channel-wise attention mechanism to increase our method's performance. Finally, we employed an auxiliary loss function to train our model. The suggested additional cost function surrogates ranking loss to increase Spearman's rank correlation coefficient while it is differentiable concerning the neural network parameters. Our method achieved superior performance in \textbf{\textit{NTIRE 2021 Perceptual Image Quality Assessment}} Challenge. The implementations of our proposed method are publicly available.
Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we propose a novel ensembling technique for deep neural networks, which is able to drastically reduce the required memory compared to alternative approaches. In particular, we propose to extract multiple sub-networks from a single, untrained neural network by solving an end-to-end optimization task combining differentiable scaling over the original architecture, with multiple regularization terms favouring the diversity of the ensemble. Since our proposal aims to detect and extract sub-structures, we call it Structured Ensemble. On a large experimental evaluation, we show that our method can achieve higher or comparable accuracy to competing methods while requiring significantly less storage. In addition, we evaluate our ensembles in terms of predictive calibration and uncertainty, showing they compare favourably with the state-of-the-art. Finally, we draw a link with the continual learning literature, and we propose a modification of our framework to handle continuous streams of tasks with a sub-linear memory cost. We compare with a number of alternative strategies to mitigate catastrophic forgetting, highlighting advantages in terms of average accuracy and memory.
Metric Entropy Limits on Recurrent Neural Network Learning of Linear Dynamical Systems
Venue: arXiv cs.LG 2021
Year: 2021

One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural networks. The purpose of this paper is to establish a result in this spirit for the approximation of general discrete-time linear dynamical systems - including time-varying systems - by recurrent neural networks (RNNs). For the subclass of linear time-invariant (LTI) systems, we devise a quantitative version of this statement. Specifically, measuring the complexity of the considered class of LTI systems through metric entropy according to [4], we show that RNNs can optimally learn - or identify in system-theory parlance - stable LTI systems. For LTI systems whose input-output relation is characterized through a difference equation, this means that RNNs can learn the difference equation from input-output traces in a metric-entropy optimal manner.
A novel method of predictive collision risk area estimation for proactive pedestrian accident prevention system in urban surveillance infrastructure
Venue: arXiv cs.CV 2021
Year: 2021

Road traffic accidents, especially vehicle pedestrian collisions in crosswalk, globally pose a severe threat to human lives and have become a leading cause of premature deaths. In order to protect such vulnerable road users from collisions, it is necessary to recognize possible conflict in advance and warn to road users, not post facto. A breakthrough for proactively preventing pedestrian collisions is to recognize pedestrian's potential risks based on vision sensors such as CCTVs. In this study, we propose a predictive collision risk area estimation system at unsignalized crosswalks. The proposed system applied trajectories of vehicles and pedestrians from video footage after preprocessing, and then predicted their trajectories by using deep LSTM networks. With use of predicted trajectories, this system can infer collision risk areas statistically, further severity of levels is divided as danger, warning, and relative safe. In order to validate the feasibility and applicability of the proposed system, we applied it and assess the severity of potential risks in two unsignalized spots in Osan city, Korea.
Modelling and Verifying BDI Agents with Bigraphs
Venue: arXiv cs.PL 2021
Year: 2021

The Belief-Desire-Intention (BDI) architecture is a popular framework for rational agents; most verification approaches are based on reasoning about implementations of BDI programming languages. We investigate an alternative approach based on reasoning about BDI agent semantics, through a model of the execution of an agent program. We employ Milner's bigraphs as the modelling framework and present an encoding for the Conceptual Agent Notation (CAN) language - a superset of AgentSpeak featuring declarative goals, concurrency, and failure recovery. We provide an encoding of the syntax and semantics of CAN agents, and give a rigorous proof that the encoding is faithful. Verification is based on the use of mainstream software tools including BigraphER, and a small case study verifying several properties of Unmanned Aerial Vehicles (UAVs) illustrates the framework in action. The executable framework is a foundational step that will enable more advanced reasoning such as plan preference, intention priorities and trade-offs, and interactions with an environment under uncertainty.
Vision based Pedestrian Potential Risk Analysis based on Automated Behavior Feature Extraction for Smart and Safe City
Venue: arXiv cs.CV 2021
Year: 2021

Despite recent advances in vehicle safety technologies, road traffic accidents still pose a severe threat to human lives and have become a leading cause of premature deaths. In particular, crosswalks present a major threat to pedestrians, but we lack dense behavioral data to investigate the risks they face. Therefore, we propose a comprehensive analytical model for pedestrian potential risk using video footage gathered by road security cameras deployed at such crossings. The proposed system automatically detects vehicles and pedestrians, calculates trajectories by frames, and extracts behavioral features affecting the likelihood of potentially dangerous scenes between these objects. Finally, we design a data cube model by using the large amount of the extracted features accumulated in a data warehouse to perform multidimensional analysis for potential risk scenes with levels of abstraction, but this is beyond the scope of this paper, and will be detailed in a future study. In our experiment, we focused on extracting the various behavioral features from multiple crosswalks, and visualizing and interpreting their behaviors and relationships among them by camera location to show how they may or may not contribute to potential risk. We validated feasibility and applicability by applying it in multiple crosswalks in Osan city, Korea.