OpenRaiser/Intern-Atlas huggingface dataset:
Fusion of Heterogeneous Friction Estimates for Traction Adaptive Motion Planning and Control
Venue: arXiv cs.RO 2021
Year: 2021

Traction adaptive motion planning and control has potential to improve an an automated vehicle's ability to avoid accident in a critical situation. However, such functionality require an accurate friction estimate for the road ahead of the vehicle that is updated in real time. Current state of the art friction estimation techniques include high accuracy local friction estimation in the presence of tire slip, as well as rough classification of the road surface ahead of the vehicle, based on forward looking camera. In this paper we show that neither of these techniques in isolation yield satisfactory behavior when deployed with traction adaptive motion planning and control functionality. However, fusion of the two provides sufficient accuracy, availability and foresight to yield near optimal behavior. To this end, we propose a fusion method based on heteroscedastic gaussian process regression, and present initial simulation based results.
On the Soundness of Infrastructure Adversaries
Venue: arXiv cs.CR 2021
Year: 2021

Companies and network operators perform risk assessment to inform policy-making, guide infrastructure investments or to comply with security standards such as ISO 27001. Due to the size and complexity of these networks, risk assessment techniques such as attack graphs or trees describe the attacker with a finite set of rules. This characterization of the attacker can easily miss attack vectors or overstate them, potentially leading to incorrect risk estimation. In this work, we propose the first methodology to justify a rule-based attacker model. Conceptually, we add another layer of abstraction on top of the symbolic model of cryptography, which reasons about protocols and abstracts cryptographic primitives. This new layer reasons about Internet-scale networks and abstracts protocols. We show, in general, how the soundness and completeness of a rule-based model can be ensured by verifying trace properties, linking soundness to safety properties and completeness to liveness properties. We then demonstrate the approach for a recently proposed threat model that quantifies the confidentiality of email communication on the Internet, including DNS, DNSSEC, and SMTP. Using off-the-shelf protocol verification tools, we discover two flaws in their threat model. After fixing them, we show that it provides symbolic soundness.
Linearizability: A Typo
Venue: arXiv cs.DC 2021
Year: 2021

Linearizability is the de facto consistency condition for concurrent objects, widely used in theory and practice. Loosely speaking, linearizability classifies concurrent executions as correct if operations on shared objects appear to take effect instantaneously during the operation execution time. This paper calls attention to a somewhat-neglected aspect of linearizability: restrictions on how pending invocations are handled, an issue that has become increasingly important for software running on systems with non-volatile main memory. Interestingly, the original published definition of linearizability includes a typo (a symbol is missing a prime) that concerns exactly this issue. In this paper we point out the typo and provide an amendment to make the definition complete. We believe that pointing this typo out rigorously and proposing a fix is important and timely.
Classifying Long Clinical Documents with Pre-trained Transformers
Venue: arXiv cs.CL 2021
Year: 2021

Automatic phenotyping is a task of identifying cohorts of patients that match a predefined set of criteria. Phenotyping typically involves classifying long clinical documents that contain thousands of tokens. At the same time, recent state-of-art transformer-based pre-trained language models limit the input to a few hundred tokens (e.g. 512 tokens for BERT). We evaluate several strategies for incorporating pre-trained sentence encoders into document-level representations of clinical text, and find that hierarchical transformers without pre-training are competitive with task pre-trained models.
Learning Group Activities from Skeletons without Individual Action Labels
Venue: arXiv cs.CV 2021
Year: 2021

To understand human behavior we must not just recognize individual actions but model possibly complex group activity and interactions. Hierarchical models obtain the best results in group activity recognition but require fine grained individual action annotations at the actor level. In this paper we show that using only skeletal data we can train a state-of-the art end-to-end system using only group activity labels at the sequence level. Our experiments show that models trained without individual action supervision perform poorly. On the other hand we show that pseudo-labels can be computed from any pre-trained feature extractor with comparable final performance. Finally our carefully designed lean pose only architecture shows highly competitive results versus more complex multimodal approaches even in the self-supervised variant.
Discovering the Rationale of Decisions: Experiments on Aligning Learning and Reasoning
Venue: arXiv cs.AI 2021
Year: 2021

In AI and law, systems that are designed for decision support should be explainable when pursuing justice. In order for these systems to be fair and responsible, they should make correct decisions and make them using a sound and transparent rationale. In this paper, we introduce a knowledge-driven method for model-agnostic rationale evaluation using dedicated test cases, similar to unit-testing in professional software development. We apply this new method in a set of machine learning experiments aimed at extracting known knowledge structures from artificial datasets from fictional and non-fictional legal settings. We show that our method allows us to analyze the rationale of black-box machine learning systems by assessing which rationale elements are learned or not. Furthermore, we show that the rationale can be adjusted using tailor-made training data based on the results of the rationale evaluation.
DialogSum: A Real-Life Scenario Dialogue Summarization Dataset
Venue: arXiv cs.CL 2021
Year: 2021

Proposal of large-scale datasets has facilitated research on deep neural models for news summarization. Deep learning can also be potentially useful for spoken dialogue summarization, which can benefit a range of real-life scenarios including customer service management and medication tracking. To this end, we propose DialogSum, a large-scale labeled dialogue summarization dataset. We conduct empirical analysis on DialogSum using state-of-the-art neural summarizers. Experimental results show unique challenges in dialogue summarization, such as spoken terms, special discourse structures, coreferences and ellipsis, pragmatics and social common sense, which require specific representation learning technologies to better deal with.
Exploring the Intrinsic Probability Distribution for Hyperspectral Anomaly Detection
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, neural network-based anomaly detection methods have attracted considerable attention in the hyperspectral remote sensing domain due to the powerful reconstruction ability compared with traditional methods. However, actual probability distribution statistics hidden in the latent space are not discovered by exploiting the reconstruction error because the probability distribution of anomalies is not explicitly modeled. To address the issue, we propose a novel probability distribution representation detector (PDRD) that explores the intrinsic distribution of both the background and the anomalies in original data for hyperspectral anomaly detection in this paper. First, we represent the hyperspectral data with multivariate Gaussian distributions from a probabilistic perspective. Then, we combine the local statistics with the obtained distributions to leverage the spatial information. Finally, the difference between the corresponding distributions of the test pixel and the average expectation of the pixels in the Chebyshev neighborhood is measured by computing the modified Wasserstein distance to acquire the detection map. We conduct the experiments on four real data sets to evaluate the performance of our proposed method. Experimental results demonstrate the accuracy and efficiency of our proposed method compared to the state-of-the-art detection methods.
Agree to Disagree: When Deep Learning Models With Identical Architectures Produce Distinct Explanations
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning of neural networks has progressively become more prominent in healthcare with models reaching, or even surpassing, expert accuracy levels. However, these success stories are tainted by concerning reports on the lack of model transparency and bias against some medical conditions or patients' sub-groups. Explainable methods are considered the gateway to alleviate many of these concerns. In this study we demonstrate that the generated explanations are volatile to changes in model training that are perpendicular to the classification task and model structure. This raises further questions about trust in deep learning models for healthcare. Mainly, whether the models capture underlying causal links in the data or just rely on spurious correlations that are made visible via explanation methods. We demonstrate that the output of explainability methods on deep neural networks can vary significantly by changes of hyper-parameters, such as the random seed or how the training set is shuffled. We introduce a measure of explanation consistency which we use to highlight the identified problems on the MIMIC-CXR dataset. We find explanations of identical models but with different training setups have a low consistency: $\approx$ 33% on average. On the contrary, kernel methods are robust against any orthogonal changes, with explanation consistency at 94%. We conclude that current trends in model explanation are not sufficient to mitigate the risks of deploying models in real life healthcare applications.
Predicting Surface Reflectance Properties of Outdoor Scenes Under Unknown Natural Illumination
Venue: arXiv cs.CV 2021
Year: 2021

Estimating and modelling the appearance of an object under outdoor illumination conditions is a complex process. Although there have been several studies on illumination estimation and relighting, very few of them focus on estimating the reflectance properties of outdoor objects and scenes. This paper addresses this problem and proposes a complete framework to predict surface reflectance properties of outdoor scenes under unknown natural illumination. Uniquely, we recast the problem into its two constituent components involving the BRDF incoming light and outgoing view directions: (i) surface points' radiance captured in the images, and outgoing view directions are aggregated and encoded into reflectance maps, and (ii) a neural network trained on reflectance maps of renders of a unit sphere under arbitrary light directions infers a low-parameter reflection model representing the reflectance properties at each surface in the scene. Our model is based on a combination of phenomenological and physics-based scattering models and can relight the scenes from novel viewpoints. We present experiments that show that rendering with the predicted reflectance properties results in a visually similar appearance to using textures that cannot otherwise be disentangled from the reflectance properties.
Threshold Martingales and the Evolution of Forecasts
Venue: arXiv cs.LG 2021
Year: 2021

This paper introduces a martingale that characterizes two properties of evolving forecast distributions. Ideal forecasts of a future event behave as martingales, sequen- tially updating the forecast to leverage the available information as the future event approaches. The threshold martingale introduced here measures the proportion of the forecast distribution lying below a threshold. In addition to being calibrated, a threshold martingale has quadratic variation that accumulates to a total determined by a quantile of the initial forecast distribution. Deviations from calibration or to- tal volatility signal problems in the underlying model. Calibration adjustments are well-known, and we augment these by introducing a martingale filter that improves volatility while guaranteeing smaller mean squared error. Thus, post-processing can rectify problems with calibration and volatility without revisiting the original forecast- ing model. We apply threshold martingales first to forecasts from simulated models and then to models that predict the winner in professional basketball games.
Fully Dynamic Set Cover via Hypergraph Maximal Matching: An Optimal Approximation Through a Local Approach
Venue: arXiv cs.DS 2021
Year: 2021

In the (fully) dynamic set cover problem, we have a collection of $m$ sets from a universe of size $n$ that undergo element insertions and deletions; the goal is to maintain an approximate set cover of the universe after each update. We give an $O(f^2)$ update time algorithm for this problem that achieves an $f$-approximation, where $f$ is the maximum number of sets that an element belongs to; under the unique games conjecture, this approximation is best possible for any fixed $f$. This is the first algorithm for dynamic set cover with approximation ratio that {exactly} matches $f$ (as opposed to {almost} $f$ in prior work), as well as the first one with runtime \emph{independent of $n,m$} (for any approximation factor of $o(f^3)$). Prior to our work, the state-of-the-art algorithms for this problem were $O(f^2)$ update time algorithms of Gupta et al. [STOC'17] and Bhattacharya et al. [IPCO'17] with $O(f^3)$ approximation, and the recent algorithm of Bhattacharya et al. [FOCS'19] with $O(f \cdot \log{n}/ε^2)$ update time and $(1+ε) \cdot f$ approximation, improving the $O(f^2 \cdot \log{n}/ε^5)$ bound of Abboud et al. [STOC'19]. The key technical ingredient of our work is an algorithm for maintaining a {maximal} matching in a dynamic hypergraph of rank $r$, where each hyperedge has at most $r$ vertices, which undergoes hyperedge insertions and deletions in $O(r^2)$ amortized update time; our algorithm is randomized, and the bound on the update time holds in expectation and with high probability. This result generalizes the maximal matching algorithm of Solomon [FOCS'16] with constant update time in ordinary graphs to hypergraphs, and is of independent merit; the previous state-of-the-art algorithms for set cover do not translate to (integral) matchings for hypergraphs, let alone a maximal one. Our quantitative result for the set cover problem is [...]
Adding Indoor Capacity Without Fiber Backhaul: A mmWave Bridge Prototype
Venue: arXiv cs.NI 2021
Year: 2021

Today, a large portion of the mobile data traffic is consumed behind the shielding walls of buildings or in the Faraday cage of trains. This renders cellular network coverage from outdoor cell sites difficult. Indoor small cells and distributed antennas along train tracks are often considered as a solution, but the cost and the need for optical fiber backhaul are often prohibitive. To alleviate this issue, we describe an out-of-band repeater that converts a sub-6 GHz cell signal from a small cell installed at a cell tower to a mmWave frequency for the fronthaul to buildings or distributed antenna sites, where the signal is downconverted to the original frequency and emitted for example inside a building. This concept does not require fiber deployment, provides backward compatibility to equipment already in use, and additional indoor capacity is gained while outdoor networks are offloaded. The architecture and hardware prototype implementation are described, and measurements are reported to demonstrate the functionality and compatibility with commercial infrastructure and mobile terminals.
BERT Busters: Outlier Dimensions that Disrupt Transformers
Venue: arXiv cs.CL 2021
Year: 2021

Multiple studies have shown that Transformers are remarkably robust to pruning. Contrary to this received wisdom, we demonstrate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer outputs (<0.0001% of model weights). In case of BERT and other pre-trained encoder Transformers, the affected component is the scaling factors and biases in the LayerNorm. The outliers are high-magnitude normalization parameters that emerge early in pre-training and show up consistently in the same dimensional position throughout the model. We show that disabling them significantly degrades both the MLM loss and the downstream task performance. This effect is observed across several BERT-family models and other popular pre-trained Transformer architectures, including BART, XLNet and ELECTRA; we also show a similar effect in GPT-2.
Optimal Virtual Network Embeddings for Tree Topologies
Venue: arXiv cs.DS 2021
Year: 2021

The performance of distributed and data-centric applications often critically depends on the interconnecting network. Applications are hence modeled as virtual networks, also accounting for resource demands on links. At the heart of provisioning such virtual networks lies the NP-hard Virtual Network Embedding Problem (VNEP): how to jointly map the virtual nodes and links onto a physical substrate network at minimum cost while obeying capacities. This paper studies the VNEP in the light of parameterized complexity. We focus on tree topology substrates, a case often encountered in practice and for which the VNEP remains NP-hard. We provide the first fixed-parameter algorithm for the VNEP with running time $O(3^r (s+r^2))$ for requests and substrates of $r$ and $s$ nodes, respectively. In a computational study our algorithm yields running time improvements in excess of 200x compared to state-of-the-art integer programming approaches. This makes it comparable in speed to the well-established ViNE heuristic while providing optimal solutions. We complement our algorithmic study with hardness results for the VNEP and related problems.
Divergence Estimation in Message Passing algorithms
Venue: arXiv cs.IT 2021
Year: 2021

Many modern imaging applications can be modeled as compressed sensing linear inverse problems. When the measurement operator involved in the inverse problem is sufficiently random, denoising Scalable Message Passing (SMP) algorithms have a potential to demonstrate high efficiency in recovering compressed data. One of the key components enabling SMP to achieve fast convergence, stability and predictable dynamics is the Onsager correction that must be updated at each iteration of the algorithm. This correction involves the denoiser's divergence that is traditionally estimated via the Black-Box Monte Carlo (BB-MC) method \cite{MC-divergence}. While the BB-MC method demonstrates satisfying accuracy of estimation, it requires executing the denoiser additional times at each iteration and might lead to a substantial increase in computational cost of the SMP algorithms. In this work we develop two Large System Limit models of the Onsager correction for denoisers operating within SMP algorithms and use these models to propose two practical classes of divergence estimators that require no additional executions of the denoiser and demonstrate similar or superior correction compared to the BB-MC method.
On Achilles Heel of Some Optical Network Designs and Performance Comparisons
Venue: arXiv cs.NI 2021
Year: 2021

This non-conventional paper represents the first attempt to uncover a possible vulnerability in some proposals for optical network designs and performance comparisons. While optical network designs and planning lie at the heart of achieving fiber capacity efficiency and/or operational efficiency, its combinatorial nature makes it computationally hard to reach optimal solutions for realistic scenarios. Therefore, the well-established way that have been taken for granted by not-so-small number of research papers is that an optimization model based on mixed integer linear programming (MILP) is first proposed and then due to the intractability of such combinatorial model, an heuristic algorithm is offered as an approximation. The solution-quality comparison between the MILP and heuristic is then carried out on small-scale instances including topologies and traffic tests to verify the efficacy of the proposed heuristic and the next step is to use such allegedly verified heuristic for optical network designs of realistic scenarios. This approach may nevertheless leave a critical vulnerability as there is no guarantee that one performs well in small tests will generalize adequately for large-scale cases, a common pitfall widely referred as the peril of extrapolation and/or overfitting. Besides, it is not uncommon that in some research works, for benchmarking purpose, the comparison between a new design proposal whose performance is obtained from on one heuristic and a reference design based on another heuristic is carried out. As the result of missing solution quality check, such performance comparison relied merely on heuristic solutions may be equally vulnerable as its results can be distorted and thus, be far from the possibly achieved zones. In this work, we pinpoint those issues and provide a realistic case study to highlight and demonstrate the impact of such vulnerabilities.
Prescriptive Process Monitoring for Cost-Aware Cycle Time Reduction
Venue: arXiv cs.LG 2021
Year: 2021

Reducing cycle time is a recurrent concern in the field of business process management. Depending on the process, various interventions may be triggered to reduce the cycle time of a case, for example, using a faster shipping service in an order-to-delivery process or giving a phone call to a customer to obtain missing information rather than waiting passively. Each of these interventions comes with a cost. This paper tackles the problem of determining if and when to trigger a time-reducing intervention in a way that maximizes the total net gain. The paper proposes a prescriptive process monitoring method that uses orthogonal random forest models to estimate the causal effect of triggering a time-reducing intervention for each ongoing case of a process. Based on this causal effect estimate, the method triggers interventions according to a user-defined policy. The method is evaluated on two real-life logs.
A Large Visual, Qualitative and Quantitative Dataset of Web Pages
Venue: arXiv cs.CV 2021
Year: 2021

The World Wide Web is not only one of the most important platforms of communication and information at present, but also an area of growing interest for scientific research. This motivates a lot of work and projects that require large amounts of data. However, there is no dataset that integrates the parameters and visual appearance of Web pages, because its collection is a costly task in terms of time and effort. With the support of various computer tools and programming scripts, we have created a large dataset of 49,438 Web pages. It consists of visual, textual and numerical data types, includes all countries worldwide, and considers a broad range of topics such as art, entertainment, economy, business, education, government, news, media, science, and environment, covering different cultural characteristics and varied design preferences. In this paper, we describe the process of collecting, debugging and publishing the final product, which is freely available. To demonstrate the usefulness of our dataset, we expose a binary classification model for detecting error Web pages, and a multi-class Web subject-based categorization, both problems using convolutional neural networks.