OpenRaiser/Intern-Atlas huggingface dataset:
Masking Modalities for Cross-modal Video Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Pre-training on large scale unlabelled datasets has shown impressive performance improvements in the fields of computer vision and natural language processing. Given the advent of large-scale instructional video datasets, a common strategy for pre-training video encoders is to use the accompanying speech as weak supervision. However, as speech is used to supervise the pre-training, it is never seen by the video encoder, which does not learn to process that modality. We address this drawback of current pre-training methods, which fail to exploit the rich cues in spoken language. Our proposal is to pre-train a video encoder using all the available video modalities as supervision, namely, appearance, sound, and transcribed speech. We mask an entire modality in the input and predict it using the other two modalities. This encourages each modality to collaborate with the others, and our video encoder learns to process appearance and audio as well as speech. We show the superior performance of our "modality masking" pre-training approach for video retrieval on the How2R, YouCook2 and Condensed Movies datasets.
Differential Flatness and Flatness Inspired Control of Aerial Manipulators based on Lagrangian Reduction
Venue: arXiv cs.RO 2021
Year: 2021

This paper shows that the dynamics of a general class of aerial manipulators, consist of an underactuated multi-rotor base with an arbitrary k-linked articulated manipulator, are differentially flat. Methods of Lagrangian Reduction under broken symmetries produce reduced equations of motion whose key variables: center-of-mass linear momentum, vehicle yaw angle, and manipulator relative joint angles become the flat outputs. Utilizing flatness theory and a second-order dynamic extension of the thrust input, we transform the mechanics of aerial manipulators to their equivalent trivial form with a valid relative degree. Using this flatness transformation, a quadratic programming-based controller is proposed within a Control Lyapunov Function (CLF-QP) framework, and its performance is verified in simulation.
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP
Venue: arXiv cs.CL 2021
Year: 2021

Meta-learning considers the problem of learning an efficient learning process that can leverage its past experience to accurately solve new tasks. However, the efficacy of meta-learning crucially depends on the distribution of tasks available for training, and this is often assumed to be known a priori or constructed from limited supervised datasets. In this work, we aim to provide task distributions for meta-learning by considering self-supervised tasks automatically proposed from unlabeled text, to enable large-scale meta-learning in NLP. We design multiple distributions of self-supervised tasks by considering important aspects of task diversity, difficulty, type, domain, and curriculum, and investigate how they affect meta-learning performance. Our analysis shows that all these factors meaningfully alter the task distribution, some inducing significant improvements in downstream few-shot accuracy of the meta-learned models. Empirically, results on 20 downstream tasks show significant improvements in few-shot learning -- adding up to +4.2% absolute accuracy (on average) to the previous unsupervised meta-learning method, and perform comparably to supervised methods on the FewRel 2.0 benchmark.
Attribute-Based Deep Periocular Recognition: Leveraging Soft Biometrics to Improve Periocular Recognition
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, periocular recognition has been developed as a valuable biometric identification approach, especially in wild environments (for example, masked faces due to COVID-19 pandemic) where facial recognition may not be applicable. This paper presents a new deep periocular recognition framework called attribute-based deep periocular recognition (ADPR), which predicts soft biometrics and incorporates the prediction into a periocular recognition algorithm to determine identity from periocular images with high accuracy. We propose an end-to-end framework, which uses several shared convolutional neural network (CNN)layers (a common network) whose output feeds two separate dedicated branches (modality dedicated layers); the first branch classifies periocular images while the second branch predicts softn biometrics. Next, the features from these two branches are fused together for a final periocular recognition. The proposed method is different from existing methods as it not only uses a shared CNN feature space to train these two tasks jointly, but it also fuses predicted soft biometric features with the periocular features in the training step to improve the overall periocular recognition performance. Our proposed model is extensively evaluated using four different publicly available datasets. Experimental results indicate that our soft biometric based periocular recognition approach outperforms other state-of-the-art methods for periocular recognition in wild environments.
FedGraph: Federated Graph Learning with Intelligent Sampling
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. However, existing work of federated learning mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular in many applications. Graph Convolutional Network (GCN) has been proposed as one of the most promising techniques for graph learning, but its federated setting has been seldom explored. In this paper, we propose FedGraph for federated graph learning among multiple computing clients, each of which holds a subgraph. FedGraph provides strong graph learning capability across clients by addressing two unique challenges. First, traditional GCN training needs feature data sharing among clients, leading to risk of privacy leakage. FedGraph solves this issue using a novel cross-client convolution operation. The second challenge is high GCN training overhead incurred by large graph size. We propose an intelligent graph sampling algorithm based on deep reinforcement learning, which can automatically converge to the optimal sampling policies that balance training speed and accuracy. We implement FedGraph based on PyTorch and deploy it on a testbed for performance evaluation. The experimental results of four popular datasets demonstrate that FedGraph significantly outperforms existing work by enabling faster convergence to higher accuracy.
A Minmax Utilization Algorithm for Network Traffic Scheduling of Industrial Robots
Venue: arXiv cs.RO 2021
Year: 2021

Emerging 5G and beyond wireless industrial virtualized networks are expected to support a significant number of robotic manipulators. Depending on the processes involved, these industrial robots might result in significant volume of multi-modal traffic that will need to traverse the network all the way to the (public/private) edge cloud, where advanced processing, control and service orchestration will be taking place. In this paper, we perform the traffic engineering by capitalizing on the underlying pseudo-deterministic nature of the repetitive processes of robotic manipulators in an industrial environment and propose an integer linear programming (ILP) model to minimize the maximum aggregate traffic in the network. The task sequence and time gap requirements are also considered in the proposed model. To tackle the curse of dimensionality in ILP, we provide a random search algorithm with quadratic time complexity. Numerical investigations reveal that the proposed scheme can reduce the peak data rate up to 53.4% compared with the nominal case where robotic manipulators operate in an uncoordinated fashion, resulting in significant improvement in the utilization of the underlying network resources.
Rational Agreement in the Presence of Crash Faults
Venue: arXiv cs.GT 2021
Year: 2021

Blockchain systems need to solve consensus despite the presence of rational users and failures. The notion of $(k,t)$-robustness has shown instrumental to list problems that cannot be solved if $k$ players are rational and $t$ players are Byzantine or act arbitrarily. What is less clear is whether one can solve such problems if the faults are benign. In this paper, we bridge the gap between games that are robust against Byzantine players and games that are robust against crash players. Our first result is an impossibility result: We show that no $(k,t)$-robust consensus protocol can solve consensus in the crash model if $k+2t\geq n$ unless there is a particular punishment strategy, called the $(k,t)$-baiting strategy. This reveals the need to introduce baiting as the act of rewarding a colluding node when betraying its coalition, to make blockchains more secure. Our second result is an equivalence relation between crash fault tolerant games and Byzantine fault tolerant games, which raises an interesting research question on the power of baiting to solve consensus. To this end, we show, on the one hand, that a $(k,t)$-robust consensus protocol becomes $(k+t,t)$-robust in the crash model. We show, on the other hand, that the existence of a $(k,t)$-robust consensus protocol in the crash model that does not make use of a baiting strategy implies the existence of a $(k-t,t)$-robust consensus protocol in the Byzantine model, with the help of cryptography.
WaveSense: Efficient Temporal Convolutions with Spiking Neural Networks for Keyword Spotting
Venue: arXiv cs.LG 2021
Year: 2021

Ultra-low power local signal processing is a crucial aspect for edge applications on always-on devices. Neuromorphic processors emulating spiking neural networks show great computational power while fulfilling the limited power budget as needed in this domain. In this work we propose spiking neural dynamics as a natural alternative to dilated temporal convolutions. We extend this idea to WaveSense, a spiking neural network inspired by the WaveNet architecture. WaveSense uses simple neural dynamics, fixed time-constants and a simple feed-forward architecture and hence is particularly well suited for a neuromorphic implementation. We test the capabilities of this model on several datasets for keyword-spotting. The results show that the proposed network beats the state of the art of other spiking neural networks and reaches near state-of-the-art performance of artificial neural networks such as CNNs and LSTMs.
Quality change: norm or exception? Measurement, Analysis and Detection of Quality Change in Wikipedia
Venue: arXiv cs.SI 2021
Year: 2021

Wikipedia has been turned into an immensely popular crowd-sourced encyclopedia for information dissemination on numerous versatile topics in the form of subscription free content. It allows anyone to contribute so that the articles remain comprehensive and updated. For enrichment of content without compromising standards, the Wikipedia community enumerates a detailed set of guidelines, which should be followed. Based on these, articles are categorized into several quality classes by the Wikipedia editors with increasing adherence to guidelines. This quality assessment task by editors is laborious as well as demands platform expertise. As a first objective, in this paper, we study evolution of a Wikipedia article with respect to such quality scales. Our results show novel non-intuitive patterns emerging from this exploration. As a second objective we attempt to develop an automated data driven approach for the detection of the early signals influencing the quality change of articles. We posit this as a change point detection problem whereby we represent an article as a time series of consecutive revisions and encode every revision by a set of intuitive features. Finally, various change point detection algorithms are used to efficiently and accurately detect the future change points. We also perform various ablation studies to understand which group of features are most effective in identifying the change points. To the best of our knowledge, this is the first work that rigorously explores English Wikipedia article quality life cycle from the perspective of quality indicators and provides a novel unsupervised page level approach to detect quality switch, which can help in automatic content monitoring in Wikipedia thus contributing significantly to the CSCW community.
Household Cloth Object Set: Fostering Benchmarking in Deformable Object Manipulation
Venue: arXiv cs.RO 2021
Year: 2021

Benchmarking of robotic manipulations is one of the open issues in robotic research. An important factor that has enabled progress in this area in the last decade is the existence of common object sets that have been shared among different research groups. However, the existing object sets are very limited when it comes to cloth-like objects that have unique particularities and challenges. This paper is a first step towards the design of a cloth object set to be distributed among research groups from the robotics cloth manipulation community. We present a set of household cloth objects and related tasks that serve to expose the challenges related to gathering such an object set and propose a roadmap to the design of common benchmarks in cloth manipulation tasks, with the intention to set the grounds for a future debate in the community that will be necessary to foster benchmarking for the manipulation of cloth-like objects. Some RGB-D and object scans are also collected as examples for the objects in relevant configurations. More details about the cloth set are shared in http://www.iri.upc.edu/groups/perception/ClothObjectSet/HouseholdClothSet.html.
Physical Channel Modeling for RIS-Empowered Wireless Networks in Sub-6 GHz Bands
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surface (RIS)-assisted communications is one of the promising candidates for next generation wireless networks by controlling the propagation environment dynamically. In this study, a channel modeling strategy for RIS-assisted wireless networks is introduced in sub-6 GHz bands by considering both far-field and near-field behaviours in transmission. We also proposed an open-source physical channel simulator for sub-6 GHz bands where operating frequency, propagation environment, terminal locations, RIS location and size can be adjusted. It is demonstrated via extensive computer simulations that an improved achievable rate performance is obtained in the presence of RISs for both near-field and far-field conditions.
MillenniumDB: A Persistent, Open-Source, Graph Database
Venue: arXiv cs.DB 2021
Year: 2021

In this systems paper, we present MillenniumDB: a novel graph database engine that is modular, persistent, and open source. MillenniumDB is based on a graph data model, which we call domain graphs, that provides a simple abstraction upon which a variety of popular graph models can be supported. The engine itself is founded on a combination of tried and tested techniques from relational data management, state-of-the-art algorithms for worst-case-optimal joins, as well as graph-specific algorithms for evaluating path queries. In this paper, we present the main design principles underlying MillenniumDB, describing the abstract graph model and query semantics supported, the concrete data model and query syntax implemented, as well as the storage, indexing, query planning and query evaluation techniques used. We evaluate MillenniumDB over real-world data and queries from the Wikidata knowledge graph, where we find that it outperforms other popular persistent graph database engines (including both enterprise and open source alternatives) that support similar query features.
A Comprehensive Survey on Nanophotonics Neural Networks
Venue: arXiv cs.ET 2021
Year: 2021

In the last years, materializations of neuromorphic circuits based on nanophotonic arrangements have been proposed, which contain complete optical circuits, laser, photodetectors, photonic crystals, optical fibers, flat waveguides, and other passive optical elements of nanostructured materials, which eliminate the time of simultaneous processing of big groups of data, taking advantage of the quantum perspective and thus highly increasing the potentials of contemporary intelligent computational systems. This article is an effort to record and study the research that has been conducted concerning the methods of development and materi-alization of neuromorphic circuits of Neural Networks of nanophotonic arrangements. In particular, an investigative study of the methods of developing nanophotonic neuromorphic processors, their originality in neuronic architectural structure, their training methods and their optimization has been realized along with the study of special issues such as optical activation functions and cost functions.
Classifying Approximation Algorithms: Understanding the APX Complexity Class
Venue: arXiv cs.CC 2021
Year: 2021

We are interested in the intersection of approximation algorithms and complexity theory, in particular focusing on the complexity class APX. Informally, APX $\subseteq$ NPO is the complexity class comprising optimization problems where the ratio $\frac{OPT(I)}{ALG(I)} \leq c$ for all instances I. We will do a deep dive into studying APX as a complexity class, in particular, investigating how researchers have defined PTAS and L reductions, as well as the notion of APX-completeness, thereby clarifying where APX lies on the polynomial hierarchy. We will discuss the relationship of this class with FPTAS, PTAS, APX, log-APX and poly-APX). We will sketch the proof that Max 3-SAT is APX-hard, and compare this complexity class in relation to $BPP$, $ZPP$ to elucidate whether randomization is powerful enough to achieve certain approximation guarantees and introduce techniques that complement the design of approximation algorithms such as through \textit{primal-dual} analysis, \textit{local search} and \textit{semi-definite programming}. Through the PCP theorem, we will explore the fundamental relationship between hardness of approximation and randomness, and will recast the way we look at the complexity class NP. We will finish by looking at the \textit{"real world"} applications of this material in Economics. Finally, we will touch upon recent breakthroughs in the Metric Travelling Salesman and asymmetric travelling salesman problem, as well original directions for future research, such as quantifying the amount of additional compute power that access to an APX oracle provides, elucidating fundamental combinatorial properties of log-APX problems and unique ways to attack the problem of whether the minimum set-cover problem is self-improvable.
SO{U}RCERER: Developer-Driven Security Testing Framework for Android Apps
Venue: arXiv cs.SE 2021
Year: 2021

Frequently advised secure development recommendations often fall short in practice for app developers. Tool-driven (e.g., using static analysis tools) approaches lack context and domain-specific requirements of an app being tested. App developers struggle to find an actionable and prioritized list of vulnerabilities from a laundry list of security warnings reported by static analysis tools. Process-driven (e.g., applying threat modeling methods) approaches require substantial resources (e.g., security testing team, budget) and security expertise, which small to medium-scale app dev teams could barely afford. To help app developers securing their apps, we propose SO{U}RCERER, a guiding framework for Android app developers for security testing. SO{U}RCERER guides developers to identify domain-specific assets of an app, detect and prioritize vulnerabilities, and mitigate those vulnerabilities based on secure development guidelines. We evaluated SO{U}RCERER with a case study on analyzing and testing 36 Android mobile money apps. We found that by following activities guided by SO{U}RCERER, an app developer could get a concise and actionable list of vulnerabilities (24-61% fewer security warnings produced by SO{U}RCERER than a standalone static analyzer), directly affecting a mobile money app's critical assets, and devise a mitigation plan. Our findings from this preliminary study indicate a viable approach to Android app security testing without being overwhelmingly complex for app developers.
Saliency detection with moving camera via background model completion
Venue: arXiv cs.CV 2021
Year: 2021

To detect saliency in video is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will lead to false negative errors. With moving camera, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises of a background modeler and the deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, although pre-trained with a specific video dataset, can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing of a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving camera. The results, obtained from the PTZ videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high ranking background subtraction methods by more than 3%.
Using Synthetic Images To Uncover Population Biases In Facial Landmarks Detection
Venue: arXiv cs.CV 2021
Year: 2021

In order to analyze a trained model performance and identify its weak spots, one has to set aside a portion of the data for testing. The test set has to be large enough to detect statistically significant biases with respect to all the relevant sub-groups in the target population. This requirement may be difficult to satisfy, especially in data-hungry applications. We propose to overcome this difficulty by generating synthetic test set. We use the face landmarks detection task to validate our proposal by showing that all the biases observed on real datasets are also seen on a carefully designed synthetic dataset. This shows that synthetic test sets can efficiently detect a model's weak spots and overcome limitations of real test set in terms of quantity and/or diversity.
Rethinking the Knowledge Distillation From the Perspective of Model Calibration
Venue: arXiv cs.CV 2021
Year: 2021

Recent years have witnessed dramatically improvements in the knowledge distillation, which can generate a compact student model for better efficiency while retaining the model effectiveness of the teacher model. Previous studies find that: more accurate teachers do not necessary make for better teachers due to the mismatch of abilities. In this paper, we aim to analysis the phenomenon from the perspective of model calibration. We found that the larger teacher model may be too over-confident, thus the student model cannot effectively imitate. While, after the simple model calibration of the teacher model, the size of the teacher model has a positive correlation with the performance of the student model.