OpenRaiser/Intern-Atlas huggingface dataset:
Virtual Axle Detector based on Analysis of Bridge Acceleration Measurements by Fully Convolutional Network
Venue: arXiv cs.CV 2022
Year: 2022

In the practical application of the Bridge Weigh-In-Motion (BWIM) methods, the position of the wheels or axles during the passage of a vehicle is in most cases a prerequisite. To avoid the use of conventional axle detectors and bridge type specific methods, we propose a novel method for axle detection through the placement of accelerometers at any point of a bridge. In order to develop a model that is as simple and comprehensible as possible, the axle detection task is implemented as a binary classification problem instead of a regression problem. The model is implemented as a Fully Convolutional Network to process signals in the form of Continuous Wavelet Transforms. This allows passages of any length to be processed in a single step with maximum efficiency while utilising multiple scales in a single evaluation. This enables our method to use acceleration signals at any location of the bridge structure serving as Virtual Axle Detectors (VADs) without being limited to specific structural types of bridges. To test the proposed method, we analysed 3787 train passages recorded on a steel trough railway bridge of a long-distance traffic line. Our results on the measurement data show that our model detects 95% of the axes, thus, 128,599 of 134,800 previously unseen axles were correctly detected. In total, 90% of the axles can be detected with a maximum spatial error of 20cm, with a maximum velocity of $v_{\mathrm{max}}=56,3~\mathrm{m/s}$. The analysis shows that our developed model can use accelerometers as VADs even under real operating conditions.
Robust Newsvendor Problem in Global Market: Stable Operation Strategy for a Two-Market Stochastic System
Venue: arXiv cs.LG 2022
Year: 2022

The global markets provide enterprises with selling opportunities and challenges in stabilizing operational strategies. From the perspective of production management, it is important to improve the profitability of an enterprise by exploiting the different timing of the selling season in different markets to develop an operational strategy that is optimized and configured on a global scale. This paper examines the above issue with an insightful model of selling the product to two markets (a primary and a secondary market) with multiple risks of changes in the market environment and nonoverlapping selling seasons. We refer to this problem as the "global robust newsvendor" problem. We provide closed-form solutions of the optimal operation strategy for demand-independent and demand-related scenarios for the above two market stochastic systems. The closed-form solutions fully reflect the influence of the relationship between supply and demand on strategy selection. We find that the demand correlation and the lack of demand information will not substantially affect the operation strategy, and the enterprise's industrial chain and supply chain remain stable. However, the reduction of inter-market tariffs or logistics costs will cause changes, and the existence of the secondary market will lead to more capacity planning in the primary market. In addition, our model explicitly considers the impact of exchange rate uncertainty on operating strategies.
Beyond Transfer Learning: Co-finetuning for Action Localisation
Venue: arXiv cs.CV 2022
Year: 2022

Transfer learning is the predominant paradigm for training deep networks on small target datasets. Models are typically pretrained on large ``upstream'' datasets for classification, as such labels are easy to collect, and then finetuned on ``downstream'' tasks such as action localisation, which are smaller due to their finer-grained annotations. In this paper, we question this approach, and propose co-finetuning -- simultaneously training a single model on multiple ``upstream'' and ``downstream'' tasks. We demonstrate that co-finetuning outperforms traditional transfer learning when using the same total amount of data, and also show how we can easily extend our approach to multiple ``upstream'' datasets to further improve performance. In particular, co-finetuning significantly improves the performance on rare classes in our downstream task, as it has a regularising effect, and enables the network to learn feature representations that transfer between different datasets. Finally, we observe how co-finetuning with public, video classification datasets, we are able to achieve state-of-the-art results for spatio-temporal action localisation on the challenging AVA and AVA-Kinetics datasets, outperforming recent works which develop intricate models.
Zero-Shot Cost Models for Distributed Stream Processing
Venue: arXiv cs.DC 2022
Year: 2022

This paper proposes a learned cost estimation model for Distributed Stream Processing Systems (DSPS) with an aim to provide accurate cost predictions of executing queries. A major premise of this work is that the proposed learned model can generalize to the dynamics of streaming workloads out-of-the-box. This means a model once trained can accurately predict performance metrics such as latency and throughput even if the characteristics of the data and workload or the deployment of operators to hardware changes at runtime. That way, the model can be used to solve tasks such as optimizing the placement of operators to minimize the end-to-end latency of a streaming query or maximize its throughput even under varying conditions. Our evaluation on a well-known DSPS, Apache Storm, shows that the model can predict accurately for unseen workloads and queries while generalizing across real-world benchmarks.
Continuous Methods : Hamiltonian Domain Translation
Venue: arXiv cs.CV 2022
Year: 2022

This paper proposes a novel approach to domain translation. Leveraging established parallels between generative models and dynamical systems, we propose a reformulation of the Cycle-GAN architecture. By embedding our model with a Hamiltonian structure, we obtain a continuous, expressive and most importantly invertible generative model for domain translation.
Decision Trees for Analyzing Influences on the Accuracy of Indoor Localization Systems
Venue: arXiv cs.RO 2022
Year: 2022

Absolute position accuracy is the key performance criterion of an Indoor Localization System (ILS). Since ILS are heterogeneous and complex cyber-physical systems, the localization accuracy depends on various influences from the environment, system configuration, and the application processes. To determine the position accuracy of a system in a reproducible, comparable, and realistic manner, these factors must be taken into account. We propose a strategy for analyzing the influences on the position accuracy of ILS using decision trees in combination with application-related or technology-related categorization. The proposed strategy is validated using empirical data from 120 experiments. The accuracy of an Ultra-Wideband and a LiDAR-based ILS was determined under different application-driven influencing factors, considering the application of autonomous mobile robots in warehouses. Finally, the opportunities and limitations of analyzing decision trees to compare system performance, find a suitable system, optimize the environment or system configuration, and understand the relevance of different influencing factors are presented.
Defense Against Multi-target Trojan Attacks
Venue: arXiv cs.CV 2022
Year: 2022

Adversarial attacks on deep learning-based models pose a significant threat to the current AI infrastructure. Among them, Trojan attacks are the hardest to defend against. In this paper, we first introduce a variation of the Badnet kind of attacks that introduces Trojan backdoors to multiple target classes and allows triggers to be placed anywhere in the image. The former makes it more potent and the latter makes it extremely easy to carry out the attack in the physical space. The state-of-the-art Trojan detection methods fail with this threat model. To defend against this attack, we first introduce a trigger reverse-engineering mechanism that uses multiple images to recover a variety of potential triggers. We then propose a detection mechanism by measuring the transferability of such recovered triggers. A Trojan trigger will have very high transferability i.e. they make other images also go to the same class. We study many practical advantages of our attack method and then demonstrate the detection performance using a variety of image datasets. The experimental results show the superior detection performance of our method over the state-of-the-arts.
Ensemble random forest filter: An alternative to the ensemble Kalman filter for inverse modeling
Venue: arXiv cs.LG 2022
Year: 2022

The ensemble random forest filter (ERFF) is presented as an alternative to the ensemble Kalman filter (EnKF) for the purpose of inverse modeling. The EnKF is a data assimilation approach that forecasts and updates parameter estimates sequentially in time as observations are being collected. The updating step is based on the experimental covariances computed from an ensemble of realizations and the updates are given as linear combinations of the differences between observations and forecasted system state values. The ERFF replaces the linear combination in the update step with a non-linear function represented by a random forest. In this way, the non-linear relationships between the parameters to be updated and the observations can be captured and a better update produced. The ERFF is demonstrated for the purpose of log-conductivity identification from piezometric head observations in a number of scenarios with varying degrees of heterogeneity (log-conductivity variances going from 1 up to 6.25 (ln m/d)2), number of realizations in the ensemble (50 or 100), and number of piezometric head observations (18 or 36). In all scenarios, the ERFF works well, being able to reconstruct the log-conductivity spatial heterogeneity while matching the observed piezometric heads at selected control points. For benchmarking purposes the ERFF is compared to the restart EnKF to find that the ERFF is superior to the EnKF for the number of ensemble realizations used (small in typical EnKF applications). Only when the number of realizations grows to 500, the restart EnKF is able to match the performance of the ERFF, albeit at triple the computational cost.
A Mask Attention Interaction and Scale Enhancement Network for SAR Ship Instance Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Most of existing synthetic aperture radar (SAR) ship in-stance segmentation models do not achieve mask interac-tion or offer limited interaction performance. Besides, their multi-scale ship instance segmentation performance is moderate especially for small ships. To solve these problems, we propose a mask attention interaction and scale enhancement network (MAI-SE-Net) for SAR ship instance segmentation. MAI uses an atrous spatial pyra-mid pooling (ASPP) to gain multi-resolution feature re-sponses, a non-local block (NLB) to model long-range spa-tial dependencies, and a concatenation shuffle attention block (CSAB) to improve interaction benefits. SE uses a content-aware reassembly of features block (CARAFEB) to generate an extra pyramid bottom-level to boost small ship performance, a feature balance operation (FBO) to improve scale feature description, and a global context block (GCB) to refine features. Experimental results on two public SSDD and HRSID datasets reveal that MAI-SE-Net outperforms the other nine competitive models, better than the suboptimal model by 4.7% detec-tion AP and 3.4% segmentation AP on SSDD and by 3.0% detection AP and 2.4% segmentation AP on HRSID.
CoCAtt: A Cognitive-Conditioned Driver Attention Dataset (Supplementary Material)
Venue: arXiv cs.CV 2022
Year: 2022

The task of driver attention prediction has drawn considerable interest among researchers in robotics and the autonomous vehicle industry. Driver attention prediction can play an instrumental role in mitigating and preventing high-risk events, like collisions and casualties. However, existing driver attention prediction models neglect the distraction state and intention of the driver, which can significantly influence how they observe their surroundings. To address these issues, we present a new driver attention dataset, CoCAtt (Cognitive-Conditioned Attention). Unlike previous driver attention datasets, CoCAtt includes per-frame annotations that describe the distraction state and intention of the driver. In addition, the attention data in our dataset is captured in both manual and autopilot modes using eye-tracking devices of different resolutions. Our results demonstrate that incorporating the above two driver states into attention modeling can improve the performance of driver attention prediction. To the best of our knowledge, this work is the first to provide autopilot attention data. Furthermore, CoCAtt is currently the largest and the most diverse driver attention dataset in terms of autonomy levels, eye tracker resolutions, and driving scenarios. CoCAtt is available for download at https://cocatt-dataset.github.io.
Flux: Liquid Types for Rust
Venue: arXiv cs.PL 2022
Year: 2022

We introduce Flux, which shows how logical refinements can work hand in glove with Rust's ownership mechanisms to yield ergonomic type-based verification of low-level pointer manipulating programs. First, we design a novel refined type system for Rust that indexes mutable locations, with pure (immutable) values that can appear in refinements, and then exploits Rust's ownership mechanisms to abstract sub-structural reasoning about locations within Rust's polymorphic type constructors, while supporting strong updates. We formalize the crucial dependency upon Rust's strong aliasing guarantees by exploiting the stacked borrows aliasing model to prove that ``well-borrowed evaluations of well-typed programs do not get stuck''. Second, we implement our type system in Flux, a plug-in to the Rust compiler that exploits the factoring of complex invariants into types and refinements to efficiently synthesize loop annotations -- including complex quantified invariants describing the contents of containers -- via liquid inference. Third, we evaluate Flux with a benchmark suite of vector manipulating programs and parts of a previously verified secure sandboxing library to demonstrate the advantages of refinement types over program logics as implemented in the state-of-the-art Prusti verifier. While Prusti's more expressive program logic can, in general, verify deep functional correctness specifications, for the lightweight but ubiquitous and important verification use-cases covered by our benchmarks, liquid typing makes verification ergonomic by slashing specification lines by a factor of two, verification time by an order of magnitude, and annotation overhead from up to 24% of code size (average 9%) to nothing at all.
Automatic Exploration of Textual Environments with Language-Conditioned Autotelic Agents
Venue: arXiv cs.AI 2022
Year: 2022

In this extended abstract we discuss the opportunities and challenges of studying intrinsically-motivated agents for exploration in textual environments. We argue that there is important synergy between text environments and autonomous agents. We identify key properties of text worlds that make them suitable for exploration by autonmous agents, namely, depth, breadth, progress niches and the ease of use of language goals; we identify drivers of exploration for such agents that are implementable in text worlds. We discuss the opportunities of using autonomous agents to make progress on text environment benchmarks. Finally we list some specific challenges that need to be overcome in this area.
Extracting Replayable Interactions from Videos of Mobile App Usage
Venue: arXiv cs.HC 2022
Year: 2022

Screen recordings of mobile apps are a popular and readily available way for users to share how they interact with apps, such as in online tutorial videos, user reviews, or as attachments in bug reports. Unfortunately, both people and systems can find it difficult to reproduce touch-driven interactions from video pixel data alone. In this paper, we introduce an approach to extract and replay user interactions in videos of mobile apps, using only pixel information in video frames. To identify interactions, we apply heuristic-based image processing and convolutional deep learning to segment screen recordings, classify the interaction in each segment, and locate the interaction point. To replay interactions on another device, we match elements on app screens using UI element detection. We evaluate the feasibility of our pixel-based approach using two datasets: the Rico mobile app dataset and a new dataset of 64 apps with both iOS and Android versions. We find that our end-to-end approach can successfully replay a majority of interactions (iOS--84.1%, Android--78.4%) on different devices, which is a step towards supporting a variety of scenarios, including automatically annotating interactions in existing videos, automated UI testing, and creating interactive app tutorials.
Generating Pseudo-labels Adaptively for Few-shot Model-Agnostic Meta-Learning
Venue: arXiv cs.LG 2022
Year: 2022

Model-Agnostic Meta-Learning (MAML) is a famous few-shot learning method that has inspired many follow-up efforts, such as ANIL and BOIL. However, as an inductive method, MAML is unable to fully utilize the information of query set, limiting its potential of gaining higher generality. To address this issue, we propose a simple yet effective method that generates psuedo-labels adaptively and could boost the performance of the MAML family. The proposed methods, dubbed Generative Pseudo-label based MAML (GP-MAML), GP-ANIL and GP-BOIL, leverage statistics of the query set to improve the performance on new tasks. Specifically, we adaptively add pseudo labels and pick samples from the query set, then re-train the model using the picked query samples together with the support set. The GP series can also use information from the pseudo query set to re-train the network during the meta-testing. While some transductive methods, such as Transductive Propagation Network (TPN), struggle to achieve this goal.
Subclasses of Class Function used to Implement Transformations of Statistical Models
Venue: arXiv cs.PL 2022
Year: 2022

A library of software for inductive inference guided by the Minimum Message Length (MML) principle was created previously. It contains various (object-oriented-) classes and subclasses of statistical Model and can be used to infer Models from given data sets in machine learning problems. Here transformations of statistical Models are considered and implemented within the library so as to have desirable properties from the object-oriented programming and mathematical points of view. The subclasses of class Function needed to do such transformations are defined.
CEG4N: Counter-Example Guided Neural Network Quantization Refinement
Venue: arXiv cs.LG 2022
Year: 2022

Neural networks are essential components of learning-based software systems. However, their high compute, memory, and power requirements make using them in low resources domains challenging. For this reason, neural networks are often quantized before deployment. Existing quantization techniques tend to degrade the network accuracy. We propose Counter-Example Guided Neural Network Quantization Refinement (CEG4N). This technique combines search-based quantization and equivalence verification: the former minimizes the computational requirements, while the latter guarantees that the network's output does not change after quantization. We evaluate CEG4N~on a diverse set of benchmarks, including large and small networks. Our technique successfully quantizes the networks in our evaluation while producing models with up to 72% better accuracy than state-of-the-art techniques.
Few-shot training LLMs for project-specific code-summarization
Venue: arXiv cs.SE 2022
Year: 2022

Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-shot and zero-shot learning: they can learn to perform a task with very few examples. Few-shotting has particular synergies in software engineering, where there are a lot of phenomena (identifier names, APIs, terminology, coding patterns) that are known to be highly project-specific. However, project-specific data can be quite limited, especially early in the history of a project; thus the few-shot learning capacity of LLMs might be very relevant. In this paper, we investigate the use few-shot training with the very large GPT (Generative Pre-trained Transformer) Codex model, and find evidence suggesting that one can significantly surpass state-of-the-art models for code-summarization, leveraging project-specific training.
A novel evaluation methodology for supervised Feature Ranking algorithms
Venue: arXiv cs.LG 2022
Year: 2022

Both in the domains of Feature Selection and Interpretable AI, there exists a desire to `rank' features based on their importance. Such feature importance rankings can then be used to either: (1) reduce the dataset size or (2) interpret the Machine Learning model. In the literature, however, such Feature Rankers are not evaluated in a systematic, consistent way. Many papers have a different way of arguing which feature importance ranker works best. This paper fills this gap, by proposing a new evaluation methodology. By making use of synthetic datasets, feature importance scores can be known beforehand, allowing more systematic evaluation. To facilitate large-scale experimentation using the new methodology, a benchmarking framework was built in Python, called fseval. The framework allows running experiments in parallel and distributed over machines on HPC systems. By integrating with an online platform called Weights and Biases, charts can be interactively explored on a live dashboard. The software was released as open-source software, and is published as a package on the PyPi platform. The research concludes by exploring one such large-scale experiment, to find the strengths and weaknesses of the participating algorithms, on many fronts.
How Do Drivers Self-Regulate their Secondary Task Engagements? The Effect of Driving Automation on Touchscreen Interactions and Glance Behavior
Venue: arXiv cs.HC 2022
Year: 2022

With ever-improving driver assistance systems and large touchscreens becoming the main in-vehicle interface, drivers are more tempted than ever to engage in distracting non-driving-related tasks. However, little research exists on how driving automation affects drivers' self-regulation when interacting with center stack touchscreens. To investigate this, we employ multilevel models on a real-world driving dataset consisting of 10,139 sequences. Our results show significant differences in drivers' interaction and glance behavior in response to varying levels of driving automation, vehicle speed, and road curvature. During partially automated driving, drivers are not only more likely to engage in secondary touchscreen tasks, but their mean glance duration toward the touchscreen also increases by 12% (Level 1) and 20% (Level 2) compared to manual driving. We further show that the effect of driving automation on drivers' self-regulation is larger than that of vehicle speed and road curvature. The derived knowledge can facilitate the safety evaluation of infotainment systems and the development of context-aware driver monitoring systems.