OpenRaiser/Intern-Atlas huggingface dataset:
Fast Passage Re-ranking with Contextualized Exact Term Matching and Efficient Passage Expansion
Venue: arXiv cs.IR 2021
Year: 2021

BERT-based information retrieval models are expensive, in both time (query latency) and computational resources (energy, hardware cost), making many of these models impractical especially under resource constraints. The reliance on a query encoder that only performs tokenization and on the pre-processing of passage representations at indexing, has allowed the recently proposed TILDE method to overcome the high query latency issue typical of BERT-based models. This however is at the expense of a lower effectiveness compared to other BERT-based re-rankers and dense retrievers. In addition, the original TILDE method is characterised by indexes with a very high memory footprint, as it expands each passage into the size of the BERT vocabulary. In this paper, we propose TILDEv2, a new model that stems from the original TILDE but that addresses its limitations. TILDEv2 relies on contextualized exact term matching with expanded passages. This requires to only store in the index the score of tokens that appear in the expanded passages (rather than all the vocabulary), thus producing indexes that are 99% smaller than those of TILDE. This matching mechanism also improves ranking effectiveness by 24%, without adding to the query latency. This makes TILDEv2 the state-of-the-art passage re-ranking method for CPU-only environments, capable of maintaining query latency below 100ms on commodity hardware.
Quad-cone-rotor: A Novel Tilt Quadrotor with Severe-fault-tolerant Ability
Venue: arXiv cs.RO 2021
Year: 2021

Conventional quadrotors received great attention in trajectory design and fault-tolerant control in these years. The direction of each thrust is perpendicular to the body because of the geometrics in mechanical design. Comparing with the conventional quadrotor, a novel quadrotor named quad-tilt-rotor brings better freedom in manipulating the thrust vector. Quad-tilt-rotor augments the additional degrees of freedom in the thrust, providing the possibility of violating the normal direction of the thrust in the conventional quadrotor. This provides the ability of greater agility in control. This paper presents a novel design of a quad-tilt-rotor (quad-cone-rotor) whose thrust can be assigned along the edge of a cone shape. Besides the inheriting merits in agile from quad-tilt-rotor, the quad-cone-rotor is expected to take fault-tolerant control in severe dynamic failure (total loss in all thrusts). We simulate the control result in a UAV simulator in SIMULINK, MATLAB.
Pruning in the Face of Adversaries
Venue: arXiv cs.LG 2021
Year: 2021

The vulnerability of deep neural networks against adversarial examples - inputs with small imperceptible perturbations - has gained a lot of attention in the research community recently. Simultaneously, the number of parameters of state-of-the-art deep learning models has been growing massively, with implications on the memory and computational resources required to train and deploy such models. One approach to control the size of neural networks is retrospectively reducing the number of parameters, so-called neural network pruning. Available research on the impact of neural network pruning on the adversarial robustness is fragmentary and often does not adhere to established principles of robustness evaluation. We close this gap by evaluating the robustness of pruned models against L-0, L-2 and L-infinity attacks for a wide range of attack strengths, several architectures, data sets, pruning methods, and compression rates. Our results confirm that neural network pruning and adversarial robustness are not mutually exclusive. Instead, sweet spots can be found that are favorable in terms of model size and adversarial robustness. Furthermore, we extend our analysis to situations that incorporate additional assumptions on the adversarial scenario and show that depending on the situation, different strategies are optimal.
Feature Stylization and Domain-aware Contrastive Learning for Domain Generalization
Venue: arXiv cs.CV 2021
Year: 2021

Domain generalization aims to enhance the model robustness against domain shift without accessing the target domain. Since the available source domains for training are limited, recent approaches focus on generating samples of novel domains. Nevertheless, they either struggle with the optimization problem when synthesizing abundant domains or cause the distortion of class semantics. To these ends, we propose a novel domain generalization framework where feature statistics are utilized for stylizing original features to ones with novel domain properties. To preserve class information during stylization, we first decompose features into high and low frequency components. Afterward, we stylize the low frequency components with the novel domain styles sampled from the manipulated statistics, while preserving the shape cues in high frequency ones. As the final step, we re-merge both components to synthesize novel domain features. To enhance domain robustness, we utilize the stylized features to maintain the model consistency in terms of features as well as outputs. We achieve the feature consistency with the proposed domain-aware supervised contrastive loss, which ensures domain invariance while increasing class discriminability. Experimental results demonstrate the effectiveness of the proposed feature stylization and the domain-aware contrastive loss. Through quantitative comparisons, we verify the lead of our method upon existing state-of-the-art methods on two benchmarks, PACS and Office-Home.
Generating Superpixels for High-resolution Images with Decoupled Patch Calibration
Venue: arXiv cs.CV 2021
Year: 2021

Superpixel segmentation has recently seen important progress benefiting from the advances in differentiable deep learning. However, the very high-resolution superpixel segmentation still remains challenging due to the expensive memory and computation cost, making the current advanced superpixel networks fail to process. In this paper, we devise Patch Calibration Networks (PCNet), aiming to efficiently and accurately implement high-resolution superpixel segmentation. PCNet follows the principle of producing high-resolution output from low-resolution input for saving GPU memory and relieving computation cost. To recall the fine details destroyed by the down-sampling operation, we propose a novel Decoupled Patch Calibration (DPC) branch for collaboratively augment the main superpixel generation branch. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. By sharing the parameters of DPC and main branches, the fine-detailed knowledge learned from high-resolution patches will be transferred to help calibrate the destroyed information. To the best of our knowledge, we make the first attempt to consider the deep-learning-based superpixel generation for high-resolution cases. To facilitate this research, we build evaluation benchmarks from two public datasets and one new constructed one, covering a wide range of diversities from fine-grained human parts to cityscapes. Extensive experiments demonstrate that our PCNet can not only perform favorably against the state-of-the-arts in the quantitative results but also improve the resolution upper bound from 3K to 5K on 1080Ti GPUs.
Probability Estimation of Uncertain Process Trace Realizations
Venue: arXiv cs.AI 2021
Year: 2021

Process mining is a scientific discipline that analyzes event data, often collected in databases called event logs. Recently, uncertain event logs have become of interest, which contain non-deterministic and stochastic event attributes that may represent many possible real-life scenarios. In this paper, we present a method to reliably estimate the probability of each of such scenarios, allowing their analysis. Experiments show that the probabilities calculated with our method closely match the true chances of occurrence of specific outcomes, enabling more trustworthy analyses on uncertain data.
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
Venue: arXiv cs.CV 2021
Year: 2021

For a given video-based Human-Object Interaction scene, modeling the spatio-temporal relationship between humans and objects are the important cue to understand the contextual information presented in the video. With the effective spatio-temporal relationship modeling, it is possible not only to uncover contextual information in each frame but also to directly capture inter-time dependencies. It is more critical to capture the position changes of human and objects over the spatio-temporal dimension when their appearance features may not show up significant changes over time. The full use of appearance features, the spatial location and the semantic information are also the key to improve the video-based Human-Object Interaction recognition performance. In this paper, Spatio-Temporal Interaction Graph Parsing Networks (STIGPN) are constructed, which encode the videos with a graph composed of human and object nodes. These nodes are connected by two types of relations: (i) spatial relations modeling the interactions between human and the interacted objects within each frame. (ii) inter-time relations capturing the long range dependencies between human and the interacted objects across frame. With the graph, STIGPN learn spatio-temporal features directly from the whole video-based Human-Object Interaction scenes. Multi-modal features and a multi-stream fusion strategy are used to enhance the reasoning capability of STIGPN. Two Human-Object Interaction video datasets, including CAD-120 and Something-Else, are used to evaluate the proposed architectures, and the state-of-the-art performance demonstrates the superiority of STIGPN.
Batch Curation for Unsupervised Contrastive Representation Learning
Venue: arXiv cs.LG 2021
Year: 2021

The state-of-the-art unsupervised contrastive visual representation learning methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data augmentations in order to construct a pretext task of instant discrimination consisting of similar and dissimilar pairs of images. Similar pairs are constructed by randomly extracting patches from the same image and applying several other transformations such as color jittering or blurring, while transformed patches from different image instances in a given batch are regarded as dissimilar pairs. We argue that this approach can result similar pairs that are \textit{semantically} dissimilar. In this work, we address this problem by introducing a \textit{batch curation} scheme that selects batches during the training process that are more inline with the underlying contrastive objective. We provide insights into what constitutes beneficial similar and dissimilar pairs as well as validate \textit{batch curation} on CIFAR10 by integrating it in the SimCLR model.
MESH: A Memory-Efficient Safe Heap for C/C++
Venue: arXiv cs.CR 2021
Year: 2021

While memory corruption bugs stemming from the use of unsafe programming languages are an old and well-researched problem, the resulting vulnerabilities still dominate real-world exploitation today. Various mitigations have been proposed to alleviate the problem, mainly in the form of language dialects, static program analysis, and code or binary instrumentation. Solutions like AdressSanitizer (ASan) and Softbound/CETS have proven that the latter approach is very promising, being able to achieve memory safety without requiring manual source code adaptions, albeit suffering substantial performance and memory overheads. While performance overhead can be seen as a flexible constraint, extensive memory overheads can be prohibitive for the use of such solutions in memory-constrained environments. To address this problem, we propose MESH, a highly memory-efficient safe heap for C/C++. With its constant, very small memory overhead (configurable up to 2 MB on x86-64) and constant complexity for pointer access checking, MESH offers efficient, byte-precise spatial and temporal memory safety for memory-constrained scenarios. Without jeopardizing the security of safe heap objects, MESH is fully compatible with existing code and uninstrumented libraries, making it practical to use in heterogeneous environments. We show the feasibility of our approach with a full LLVM-based prototype supporting both major architectures, i.e., x86-64 and ARM64, in a Linux runtime environment. Our prototype evaluation shows that, compared to ASan and Softbound/CETS, MESH can achieve huge memory savings while preserving similar execution performance.
On the Future of Cloud Engineering
Venue: arXiv cs.DC 2021
Year: 2021

Ever since the commercial offerings of the Cloud started appearing in 2006, the landscape of cloud computing has been undergoing remarkable changes with the emergence of many different types of service offerings, developer productivity enhancement tools, and new application classes as well as the manifestation of cloud functionality closer to the user at the edge. The notion of utility computing, however, has remained constant throughout its evolution, which means that cloud users always seek to save costs of leasing cloud resources while maximizing their use. On the other hand, cloud providers try to maximize their profits while assuring service-level objectives of the cloud-hosted applications and keeping operational costs low. All these outcomes require systematic and sound cloud engineering principles. The aim of this paper is to highlight the importance of cloud engineering, survey the landscape of best practices in cloud engineering and its evolution, discuss many of the existing cloud engineering advances, and identify both the inherent technical challenges and research opportunities for the future of cloud computing in general and cloud engineering in particular.
Programming-By-Example by Programming-By-Example: Synthesis of Looping Programs
Venue: arXiv cs.PL 2021
Year: 2021

Program synthesis has seen many new applications in recent years, in large part thanks to the introduction of SyGuS. However, no existing SyGuS solvers have support for synthesizing recursive functions. We introduce an multi-phase algorithm for the synthesis of recursive ``looplike'' programs in SyGuS for programming-by-example. We solve constraints individually and treat them as ``unrolled`` examples of how a recursive program would behave, and solve for the generalized recursive solution. Our approach is modular and supports any SyGuS Solver.
Accurate 3D frequency-domain seismic wave modeling with the wavelength-adaptive 27-point finite-difference stencil: a tool for full waveform inversion
Venue: arXiv cs.CE 2021
Year: 2021

Efficient frequency-domain Full Waveform Inversion (FWI) of long-offset/wide-azimuth node data can be designed with a few discrete frequencies. However, 3D frequency-domain seismic modeling remains challenging since it requires solving a large and sparse linear indefinite system per frequency. When such systems are solved with direct methods or hybrid direct/iterative solvers, based upon domain decomposition preconditioner, finite-difference stencils on regular Cartesian grids should be designed to conciliate compactness and accuracy, the former being necessary to mitigate the fill-in induced by the Lower-Upper (LU) factorization. Compactness is classically implemented by combining several second-order accurate stencils covering the eight cells surrounding the collocation point, leading to the so-called 27-point stencil. Accuracy is obtained by applying optimal weights on the different stiffness and consistent mass matrices such that numerical dispersion is jointly minimized for several number of grid points per wavelength ($G$). However, with this approach, the same weights are used at each collocation point, leading to suboptimal accuracy in heterogeneous media. In this study, we propose a straightforward recipe to improve the accuracy of the 27-point stencil. First, we finely tabulate the values of $G$ covering the range of wavelengths spanned by the subsurface model and the frequency. Then, we estimate with a classical dispersion analysis in homogeneous media the corresponding table of optimal weights that minimize dispersion for each $G$ treated separately. We however apply a Tikhonov regularization to guarantee smooth variation of the weights with $G$. Finally, we build the impedance matrix by selecting the optimal weights at each collocation point according to the local wavelength, hence leading to a wavelength-adaptive stencil.
FSNet: A Failure Detection Framework for Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Semantic segmentation is an important task that helps autonomous vehicles understand their surroundings and navigate safely. During deployment, even the most mature segmentation models are vulnerable to various external factors that can degrade the segmentation performance with potentially catastrophic consequences for the vehicle and its surroundings. To address this issue, we propose a failure detection framework to identify pixel-level misclassification. We do so by exploiting internal features of the segmentation model and training it simultaneously with a failure detection network. During deployment, the failure detector can flag areas in the image where the segmentation model have failed to segment correctly. We evaluate the proposed approach against state-of-the-art methods and achieve 12.30%, 9.46%, and 9.65% performance improvement in the AUPR-Error metric for Cityscapes, BDD100K, and Mapillary semantic segmentation datasets.
Dynamic Difficulty Adjustment in Virtual Reality Exergames through Experience-driven Procedural Content Generation
Venue: arXiv cs.HC 2021
Year: 2021

Virtual Reality (VR) games that feature physical activities have been shown to increase players' motivation to do physical exercise. However, for such exercises to have a positive healthcare effect, they have to be repeated several times a week. To maintain player motivation over longer periods of time, games often employ Dynamic Difficulty Adjustment (DDA) to adapt the game's challenge according to the player's capabilities. For exercise games, this is mostly done by tuning specific in-game parameters like the speed of objects. In this work, we propose to use experience-driven Procedural Content Generation for DDA in VR exercise games by procedurally generating levels that match the player's current capabilities. Not only finetuning specific parameters but creating completely new levels has the potential to decrease repetition over longer time periods and allows for the simultaneous adaptation of the cognitive and physical challenge of the exergame. As a proof-of-concept, we implement an initial prototype in which the player must traverse a maze that includes several exercise rooms, whereby the generation of the maze is realized by a neural network. Passing those exercise rooms requires the player to perform physical activities. To match the player's capabilities, we use Deep Reinforcement Learning to adjust the structure of the maze and to decide which exercise rooms to include in the maze. We evaluate our prototype in an exploratory user study utilizing both biodata and subjective questionnaires.
Client Selection Approach in Support of Clustered Federated Learning over Wireless Edge Networks
Venue: arXiv cs.DC 2021
Year: 2021

Clustered Federated Multitask Learning (CFL) was introduced as an efficient scheme to obtain reliable specialized models when data is imbalanced and distributed in a non-i.i.d. (non-independent and identically distributed) fashion amongst clients. While a similarity measure metric, like the cosine similarity, can be used to endow groups of the client with a specialized model, this process can be arduous as the server should involve all clients in each of the federated learning rounds. Therefore, it is imperative that a subset of clients is selected periodically due to the limited bandwidth and latency constraints at the network edge. To this end, this paper proposes a new client selection algorithm that aims to accelerate the convergence rate for obtaining specialized machine learning models that achieve high test accuracies for all client groups. Specifically, we introduce a client selection approach that leverages the devices' heterogeneity to schedule the clients based on their round latency and exploits the bandwidth reuse for clients that consume more time to update the model. Then, the server performs model averaging and clusters the clients based on predefined thresholds. When a specific cluster reaches a stationary point, the proposed algorithm uses a greedy scheduling algorithm for that group by selecting the clients with less latency to update the model. Extensive experiments show that the proposed approach lowers the training time and accelerates the convergence rate by up to 50% while imbuing each client with a specialized model that is fit for its local data distribution.
Wisdom of (Binned) Crowds: A Bayesian Stratification Paradigm for Crowd Counting
Venue: arXiv cs.CV 2021
Year: 2021

Datasets for training crowd counting deep networks are typically heavy-tailed in count distribution and exhibit discontinuities across the count range. As a result, the de facto statistical measures (MSE, MAE) exhibit large variance and tend to be unreliable indicators of performance across the count range. To address these concerns in a holistic manner, we revise processes at various stages of the standard crowd counting pipeline. To enable principled and balanced minibatch sampling, we propose a novel smoothed Bayesian sample stratification approach. We propose a novel cost function which can be readily incorporated into existing crowd counting deep networks to encourage strata-aware optimization. We analyze the performance of representative crowd counting approaches across standard datasets at per strata level and in aggregate. We analyze the performance of crowd counting approaches across standard datasets and demonstrate that our proposed modifications noticeably reduce error standard deviation. Our contributions represent a nuanced, statistically balanced and fine-grained characterization of performance for crowd counting approaches. Code, pretrained models and interactive visualizations can be viewed at our project page https://deepcount.iiit.ac.in/
EqGNN: Equalized Node Opportunity in Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Graph neural networks (GNNs), has been widely used for supervised learning tasks in graphs reaching state-of-the-art results. However, little work was dedicated to creating unbiased GNNs, i.e., where the classification is uncorrelated with sensitive attributes, such as race or gender. Some ignore the sensitive attributes or optimize for the criteria of statistical parity for fairness. However, it has been shown that neither approaches ensure fairness, but rather cripple the utility of the prediction task. In this work, we present a GNN framework that allows optimizing representations for the notion of Equalized Odds fairness criteria. The architecture is composed of three components: (1) a GNN classifier predicting the utility class, (2) a sampler learning the distribution of the sensitive attributes of the nodes given their labels. It generates samples fed into a (3) discriminator that discriminates between true and sampled sensitive attributes using a novel "permutation loss" function. Using these components, we train a model to neglect information regarding the sensitive attribute only with respect to its label. To the best of our knowledge, we are the first to optimize GNNs for the equalized odds criteria. We evaluate our classifier over several graph datasets and sensitive attributes and show our algorithm reaches state-of-the-art results.