OpenRaiser/Intern-Atlas huggingface dataset:
Towards Citizen Science for Smart Cities: A Framework for a Collaborative Game of Bird Call Recognition Based on Internet of Sound Practices
Venue: arXiv cs.SD 2021
Year: 2021

Citizen Science aims to engage people in research activities on important issues related to their well-being. Smart Cities aim to provide them with services that improve the quality of their life. Both concepts have seen significant growth in the last years, and can be further enhanced by combining their purposes with IoT technologies that allow for dynamic and large-scale communication and interaction. However, exciting and retaining the interest of participants is a key factor for such initiatives. In this paper we suggest that engagement in Citizen Science projects applied on Smart Cities infrastructure can be enhanced through contextual and structural game elements realized through augmented audio interactive mechanisms. Our inter-disciplinary framework is described through the paradigm of a collaborative bird call recognition game, in which users collect and submit audio data, which are then classified and used for augmenting physical space with virtual soundscape maps. We discuss the Playful Learning, Internet of Audio Things, and Bird Monitoring principles that shaped the design of our paradigm, and analyze its potential technical implementation.
Collaborative Label Correction via Entropy Thresholding
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks (DNNs) have the capacity to fit extremely noisy labels nonetheless they tend to learn data with clean labels first and then memorize those with noisy labels. We examine this behavior in light of the Shannon entropy of the predictions and demonstrate the low entropy predictions determined by a given threshold are much more reliable as the supervision than the original noisy labels. It also shows the advantage in maintaining more training samples than previous methods. Then, we power this entropy criterion with the Collaborative Label Correction (CLC) framework to further avoid undesired local minimums of the single network. A range of experiments have been conducted on multiple benchmarks with both synthetic and real-world settings. Extensive results indicate that our CLC outperforms several state-of-the-art methods.
Knowledge Distillation By Sparse Representation Matching
Venue: arXiv cs.CV 2021
Year: 2021

Knowledge Distillation refers to a class of methods that transfers the knowledge from a teacher network to a student network. In this paper, we propose Sparse Representation Matching (SRM), a method to transfer intermediate knowledge obtained from one Convolutional Neural Network (CNN) to another by utilizing sparse representation learning. SRM first extracts sparse representations of the hidden features of the teacher CNN, which are then used to generate both pixel-level and image-level labels for training intermediate feature maps of the student network. We formulate SRM as a neural processing block, which can be efficiently optimized using stochastic gradient descent and integrated into any CNN in a plug-and-play manner. Our experiments demonstrate that SRM is robust to architectural differences between the teacher and student networks, and outperforms other KD techniques across several datasets.
SRA-LSTM: Social Relationship Attention LSTM for Human Trajectory Prediction
Venue: arXiv cs.CV 2021
Year: 2021

Pedestrian trajectory prediction for surveillance video is one of the important research topics in the field of computer vision and a key technology of intelligent surveillance systems. Social relationship among pedestrians is a key factor influencing pedestrian walking patterns but was mostly ignored in the literature. Pedestrians with different social relationships play different roles in the motion decision of target pedestrian. Motivated by this idea, we propose a Social Relationship Attention LSTM (SRA-LSTM) model to predict future trajectories. We design a social relationship encoder to obtain the representation of their social relationship through the relative position between each pair of pedestrians. Afterwards, the social relationship feature and latent movements are adopted to acquire the social relationship attention of this pair of pedestrians. Social interaction modeling is achieved by utilizing social relationship attention to aggregate movement information from neighbor pedestrians. Experimental results on two public walking pedestrian video datasets (ETH and UCY), our model achieves superior performance compared with state-of-the-art methods. Contrast experiments with other attention methods also demonstrate the effectiveness of social relationship attention.
Smart Scribbles for Image Mating
Venue: arXiv cs.CV 2021
Year: 2021

Image matting is an ill-posed problem that usually requires additional user input, such as trimaps or scribbles. Drawing a fne trimap requires a large amount of user effort, while using scribbles can hardly obtain satisfactory alpha mattes for non-professional users. Some recent deep learning-based matting networks rely on large-scale composite datasets for training to improve performance, resulting in the occasional appearance of obvious artifacts when processing natural images. In this article, we explore the intrinsic relationship between user input and alpha mattes and strike a balance between user effort and the quality of alpha mattes. In particular, we propose an interactive framework, referred to as smart scribbles, to guide users to draw few scribbles on the input images to produce high-quality alpha mattes. It frst infers the most informative regions of an image for drawing scribbles to indicate different categories (foreground, background, or unknown) and then spreads these scribbles (i.e., the category labels) to the rest of the image via our well-designed two-phase propagation. Both neighboring low-level afnities and high-level semantic features are considered during the propagation process. Our method can be optimized without large-scale matting datasets and exhibits more universality in real situations. Extensive experiments demonstrate that smart scribbles can produce more accurate alpha mattes with reduced additional input, compared to the state-of-the-art matting methods.
Arbitrary Length k-Anonymous Dining-Cryptographers Communication
Venue: arXiv cs.NI 2021
Year: 2021

Dining-cryptographers networks (DCN) can achieve information-theoretical privacy. Unfortunately, they are not well suited for peer-to-peer networks as they are used in blockchain applications to disseminate transactions and blocks among participants. In previous but preliminary work, we proposed a threephase approach with an initial phase based on a DCN with a group size of k while later phases take care of the actual broadcast within a peer-to-peer network. This paper describes our DCN protocol in detail and adds a performance evaluation powered by our proof-of-concept implementation. Our contributions are (i) an extension of the DCN protocol by von Ahn for fair delivery of arbitrarily long messages sent by potentially multiple senders, (ii) a privacy and security analysis of this extension, (iii) various performance optimisation especially for best-case operation, and (iv) a performance evaluation. The latter uses a latency of 100 ms and a bandwidth limit of 50 Mbit/s between participants. The interquartile range of the largest test of the highly secured version took 35s+-1.25s for a full run. All tests of the optimized common-case mode show the dissemination of a message within 0.5s+-0.1s. These results compare favourably to previously established protocols for k-anonymous transmission of fixed size messages, outperforming the original protocol for messages as small as 2 KiB.
Facial expression and attributes recognition based on multi-task learning of lightweight neural networks
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, the multi-task learning of lightweight convolutional neural networks is studied for face identification and classification of facial attributes (age, gender, ethnicity) trained on cropped faces without margins. The necessity to fine-tune these networks to predict facial expressions is highlighted. Several models are presented based on MobileNet, EfficientNet and RexNet architectures. It was experimentally demonstrated that they lead to near state-of-the-art results in age, gender and race recognition on the UTKFace dataset and emotion classification on the AffectNet dataset. Moreover, it is shown that the usage of the trained models as feature extractors of facial regions in video frames leads to 4.5% higher accuracy than the previously known state-of-the-art single models for the AFEW and the VGAF datasets from the EmotiW challenges. The models and source code are publicly available at https://github.com/HSE-asavchenko/face-emotion-recognition.
iCurb: Imitation Learning-based Detection of Road Curbs using Aerial Images for Autonomous Driving
Venue: arXiv cs.CV 2021
Year: 2021

Detection of road curbs is an essential capability for autonomous driving. It can be used for autonomous vehicles to determine drivable areas on roads. Usually, road curbs are detected on-line using vehicle-mounted sensors, such as video cameras and 3-D Lidars. However, on-line detection using video cameras may suffer from challenging illumination conditions, and Lidar-based approaches may be difficult to detect far-away road curbs due to the sparsity issue of point clouds. In recent years, aerial images are becoming more and more worldwide available. We find that the visual appearances between road areas and off-road areas are usually different in aerial images, so we propose a novel solution to detect road curbs off-line using aerial images. The input to our method is an aerial image, and the output is directly a graph (i.e., vertices and edges) representing road curbs. To this end, we formulate the problem as an imitation learning problem, and design a novel network and an innovative training strategy to train an agent to iteratively find the road-curb graph. The experimental results on a public dataset confirm the effectiveness and superiority of our method. This work is accompanied with a demonstration video and a supplementary document at https://tonyxuqaq.github.io/iCurb/.
Camouflaged Instance Segmentation In-The-Wild: Dataset, Method, and Benchmark Suite
Venue: arXiv cs.CV 2021
Year: 2021

This paper pushes the envelope on decomposing camouflaged regions in an image into meaningful components, namely, camouflaged instances. To promote the new task of camouflaged instance segmentation of in-the-wild images, we introduce a dataset, dubbed CAMO++, that extends our preliminary CAMO dataset (camouflaged object segmentation) in terms of quantity and diversity. The new dataset substantially increases the number of images with hierarchical pixel-wise ground truths. We also provide a benchmark suite for the task of camouflaged instance segmentation. In particular, we present an extensive evaluation of state-of-the-art instance segmentation methods on our newly constructed CAMO++ dataset in various scenarios. We also present a camouflage fusion learning (CFL) framework for camouflaged instance segmentation to further improve the performance of state-of-the-art methods. The dataset, model, evaluation suite, and benchmark will be made publicly available on our project page: https://sites.google.com/view/ltnghia/research/camo_plus_plus
Millimeter Wave and Sub-Terahertz Spatial Statistical Channel Model for an Indoor Office Building
Venue: arXiv cs.IT 2021
Year: 2021

Millimeter-wave (mmWave) and sub-Terahertz (THz) frequencies are expected to play a vital role in 6G wireless systems and beyond due to the vast available bandwidth of many tens of GHz. This paper presents an indoor 3-D spatial statistical channel model for mmWave and sub-THz frequencies based on extensive radio propagation measurements at 28 and 140 GHz conducted in an indoor office environment from 2014 to 2020. Omnidirectional and directional path loss models and channel statistics such as the number of time clusters, cluster delays, and cluster powers were derived from over 15,000 measured power delay profiles. The resulting channel statistics show that the number of time clusters follows a Poisson distribution and the number of subpaths within each cluster follows a composite exponential distribution for both LOS and NLOS environments at 28 and 140 GHz. This paper proposes a unified indoor statistical channel model for mmWave and sub-Terahertz frequencies following the mathematical framework of the previous outdoor NYUSIM channel models. A corresponding indoor channel simulator is developed, which can recreate 3-D omnidirectional, directional, and multiple input multiple output (MIMO) channels for arbitrary mmWave and sub-THz carrier frequency up to 150 GHz, signal bandwidth, and antenna beamwidth. The presented statistical channel model and simulator will guide future air-interface, beamforming, and transceiver designs for 6G and beyond.
Efficient Exploration of Interesting Aggregates in RDF Graphs
Venue: arXiv cs.DB 2021
Year: 2021

As large Open Data are increasingly shared as RDF graphs today, there is a growing demand to help users discover the most interesting facets of a graph, which are often hard to grasp without automatic tools. We consider the problem of automatically identifying the k most interesting aggregate queries that can be evaluated on an RDF graph, given an integer k and a user-specified interestingness function. Our problem departs from analytics in relational data warehouses in that (i) in an RDF graph we are not given but we must identify the facts, dimensions, and measures of candidate aggregates; (ii) the classical approach to efficiently evaluating multiple aggregates breaks in the face of multi-valued dimensions in RDF data. In this work, we propose an extensible end-to-end framework that enables the identification and evaluation of interesting aggregates based on a new RDF-compatible one-pass algorithm for efficiently evaluating a lattice of aggregates and a novel early-stop technique (with probabilistic guarantees) that can prune uninteresting aggregates. Experiments using both real and synthetic graphs demonstrate the ability of our framework to find interesting aggregates in a large search space, the efficiency of our algorithms (with up to 2.9x speedup over a similar pipeline based on existing algorithms), and scalability as the data size and complexity grow.
Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability
Venue: arXiv cs.CL 2021
Year: 2021

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.
SkiffOS: Minimal Cross-compiled Linux for Embedded Containers
Venue: arXiv cs.SE 2021
Year: 2021

Embedded Linux processors are increasingly used for real-time computing tasks such as robotics and Internet of Things (IoT). These applications require robust and reproducible behavior from the host OS, commonly achieved through immutable firmware stored in read-only memory. SkiffOS addresses these requirements with a minimal cross-compiled GNU/Linux system optimized for hosting containerized distributions and applications, and a configuration layering system for the Buildroot embedded cross-compiler tool which automatically re-targets system configurations to any platform or device. This approach cleanly separates the hardware support from the applications. The host system and containers are independently upgraded and backed-up over-the-air (OTA).
Investigating Design Anti-pattern and Design Pattern Mutations and Their Change- and Fault-proneness
Venue: arXiv cs.SE 2021
Year: 2021

During software evolution, inexperienced developers may introduce design anti-patterns when they modify their software systems to fix bugs or to add new functionalities based on changes in requirements. Developers may also use design patterns to promote software quality or as a possible cure for some design anti-patterns. Thus, design patterns and design anti-patterns are introduced, removed, and mutated from one another by developers. Many studies investigated the evolution of design patterns and design anti-patterns and their impact on software development. However, they investigated design patterns or design anti-patterns in isolation and did not consider their mutations and the impact of these mutations on software quality. Therefore, we report our study of bidirectional mutations between design patterns and design anti-patterns and the impacts of these mutations on software change- and fault-proneness. We analyzed snapshots of seven Java software systems with diverse sizes, evolution histories, and application domains. We built Markov models to capture the probability of occurrences of the different design patterns and design anti-patterns mutations. Results from our study show that (1) design patterns and design anti-patterns mutate into other design patterns and/or design anti-patterns. They also show that (2) some change types primarily trigger mutations of design patterns and design anti-patterns (renaming and changes to comments, declarations, and operators), and (3) some mutations of design anti-patterns and design patterns are more faulty in specific contexts. These results provide important insights into the evolution of design patterns and design anti-patterns and its impact on the change- and fault-proneness of software systems.
Generalized Conflict-directed Search for Optimal Ordering Problems
Venue: arXiv cs.AI 2021
Year: 2021

Solving planning and scheduling problems for multiple tasks with highly coupled state and temporal constraints is notoriously challenging. An appealing approach to effectively decouple the problem is to judiciously order the events such that decisions can be made over sequences of tasks. As many problems encountered in practice are over-constrained, we must instead find relaxed solutions in which certain requirements are dropped. This motivates a formulation of optimality with respect to the costs of relaxing constraints and the problem of finding an optimal ordering under which this relaxing cost is minimum. In this paper, we present Generalized Conflict-directed Ordering (GCDO), a branch-and-bound ordering method that generates an optimal total order of events by leveraging the generalized conflicts of both inconsistency and suboptimality from sub-solvers for cost estimation and solution space pruning. Due to its ability to reason over generalized conflicts, GCDO is much more efficient in finding high-quality total orders than the previous conflict-directed approach CDITO. We demonstrate this by benchmarking on temporal network configuration problems, which involves managing networks over time and makes necessary tradeoffs between network flows against CDITO and Mixed Integer-Linear Programing (MILP). Our algorithm is able to solve two orders of magnitude more benchmark problems to optimality and twice the problems compared to CDITO and MILP within a runtime limit, respectively.
Ultra-Reliable Indoor Millimeter Wave Communications using Multiple Artificial Intelligence-Powered Intelligent Surfaces
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, a novel framework for guaranteeing ultra-reliable millimeter wave (mmW) communications using multiple artificial intelligence (AI)-enabled reconfigurable intelligent surfaces (RISs) is proposed. The use of multiple AI-powered RISs allows changing the propagation direction of the signals transmitted from a mmW access point (AP) thereby improving coverage particularly for non-line-of-sight (NLoS) areas. However, due to the possibility of highly stochastic blockage over mmW links, designing an intelligent controller to jointly optimize the mmW AP beam and RIS phase shifts is a daunting task. In this regard, first, a parametric risk-sensitive episodic return is proposed to maximize the expected bit rate and mitigate the risk of mmW link blockage. Then, a closed-form approximation of the policy gradient of the risk-sensitive episodic return is analytically derived. Next, the problem of joint beamforming for mmW AP and phase shift control for mmW RISs is modeled as an identical payoff stochastic game within a cooperative multi-agent environment, in which the agents are the mmW AP and the RISs. Two centralized and distributed controllers are proposed to control the policies of the mmW AP and RISs. To directly find an optimal solution, the parametric functional-form policies for these controllers are modeled using deep recurrent neural networks (RNNs). Simulation results show that the error between policies of the optimal and the RNN-based controllers is less than 1.5%. Moreover, the variance of the achievable rates resulting from the deep RNN-based controllers is 60% less than the variance of the risk-averse baseline.
Planning for Safe Abortable Overtaking Maneuvers in Autonomous Driving
Venue: arXiv cs.RO 2021
Year: 2021

Overtaking is one of the most challenging tasks in driving, and the current solutions to autonomous overtaking are limited to simple and static scenarios. In this paper, we present a method for behaviour and trajectory planning for safe autonomous overtaking. The proposed method optimizes the trajectory by simultaneously enforcing safety and minimizing intrusion onto the adjacent lane. Furthermore, the method allows the overtaking to be aborted, enabling the autonomous vehicle to merge back in the lane, if safety is compromised, because of e.g. traffic in opposing direction appearing during the maneuver execution. A finite state machine is used to select an appropriate maneuver at each time, and a combination of safe and reachable sets is used to iteratively generate intermediate reference targets based on the current maneuver. A nonlinear model predictive controller then plans dynamically feasible and collision-free trajectories to these intermediate reference targets. Simulation experiments demonstrate that the combination of intermediate reference generation and model predictive control is able to handle multiple behaviors, including following a lead vehicle, overtaking and aborting the overtake, within a single framework.