OpenRaiser/Intern-Atlas huggingface dataset:
Learning with Multiclass AUC: Theory and Algorithms
Venue: arXiv cs.LG 2021
Year: 2021

The Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. The vast majority of existing AUC-optimization-based machine learning methods only focus on binary-class cases, while leaving the multiclass cases unconsidered. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics. Our foundation is based on the M metric, which is a well-known multiclass extension of AUC. We first pay a revisit to this metric, showing that it could eliminate the imbalance issue from the minority class pairs. Motivated by this, we propose an empirical surrogate risk minimization framework to approximately optimize the M metric. Theoretically, we show that: (i) optimizing most of the popular differentiable surrogate losses suffices to reach the Bayes optimal scoring function asymptotically; (ii) the training framework enjoys an imbalance-aware generalization error bound, which pays more attention to the bottleneck samples of minority classes compared with the traditional $O(\sqrt{1/N})$ result. Practically, to deal with the low scalability of the computational operations, we propose acceleration methods for three popular surrogate loss functions, including the exponential loss, squared loss, and hinge loss, to speed up loss and gradient evaluations. Finally, experimental results on 11 real-world datasets demonstrate the effectiveness of our proposed framework.
Squeeze-Excitation Convolutional Recurrent Neural Networks for Audio-Visual Scene Classification
Venue: arXiv cs.MM 2021
Year: 2021

The use of multiple and semantically correlated sources can provide complementary information to each other that may not be evident when working with individual modalities on their own. In this context, multi-modal models can help producing more accurate and robust predictions in machine learning tasks where audio-visual data is available. This paper presents a multi-modal model for automatic scene classification that exploits simultaneously auditory and visual information. The proposed approach makes use of two separate networks which are respectively trained in isolation on audio and visual data, so that each network specializes in a given modality. The visual subnetwork is a pre-trained VGG16 model followed by a bidiretional recurrent layer, while the residual audio subnetwork is based on stacked squeeze-excitation convolutional blocks trained from scratch. After training each subnetwork, the fusion of information from the audio and visual streams is performed at two different stages. The early fusion stage combines features resulting from the last convolutional block of the respective subnetworks at different time steps to feed a bidirectional recurrent structure. The late fusion stage combines the output of the early fusion stage with the independent predictions provided by the two subnetworks, resulting in the final prediction. We evaluate the method using the recently published TAU Audio-Visual Urban Scenes 2021, which contains synchronized audio and video recordings from 12 European cities in 10 different scene classes. The proposed model has been shown to provide an excellent trade-off between prediction performance (86.5%) and system complexity (15M parameters) in the evaluation results of the DCASE 2021 Challenge.
C^3Net: End-to-End deep learning for efficient real-time visual active camera control
Venue: arXiv cs.CV 2021
Year: 2021

The need for automated real-time visual systems in applications such as smart camera surveillance, smart environments, and drones necessitates the improvement of methods for visual active monitoring and control. Traditionally, the active monitoring task has been handled through a pipeline of modules such as detection, filtering, and control. However, such methods are difficult to jointly optimize and tune their various parameters for real-time processing in resource constraint systems. In this paper a deep Convolutional Camera Controller Neural Network is proposed to go directly from visual information to camera movement to provide an efficient solution to the active vision problem. It is trained end-to-end without bounding box annotations to control a camera and follow multiple targets from raw pixel values. Evaluation through both a simulation framework and real experimental setup, indicate that the proposed solution is robust to varying conditions and able to achieve better monitoring performance than traditional approaches both in terms of number of targets monitored as well as in effective monitoring time. The advantage of the proposed approach is that it is computationally less demanding and can run at over 10 FPS (~4x speedup) on an embedded smart camera providing a practical and affordable solution to real-time active monitoring.
Bayesian Autoencoders for Drift Detection in Industrial Environments
Venue: arXiv cs.LG 2021
Year: 2021

Autoencoders are unsupervised models which have been used for detecting anomalies in multi-sensor environments. A typical use includes training a predictive model with data from sensors operating under normal conditions and using the model to detect anomalies. Anomalies can come either from real changes in the environment (real drift) or from faulty sensory devices (virtual drift); however, the use of Autoencoders to distinguish between different anomalies has not yet been considered. To this end, we first propose the development of Bayesian Autoencoders to quantify epistemic and aleatoric uncertainties. We then test the Bayesian Autoencoder using a real-world industrial dataset for hydraulic condition monitoring. The system is injected with noise and drifts, and we have found the epistemic uncertainty to be less sensitive to sensor perturbations as compared to the reconstruction loss. By observing the reconstructed signals with the uncertainties, we gain interpretable insights, and these uncertainties offer a potential avenue for distinguishing real and virtual drifts.
Towards Neural Schema Alignment for OpenStreetMap and Knowledge Graphs
Venue: arXiv cs.LG 2021
Year: 2021

OpenStreetMap (OSM) is one of the richest openly available sources of volunteered geographic information. Although OSM includes various geographical entities, their descriptions are highly heterogeneous, incomplete, and do not follow any well-defined ontology. Knowledge graphs can potentially provide valuable semantic information to enrich OSM entities. However, interlinking OSM entities with knowledge graphs is inherently difficult due to the large, heterogeneous, ambiguous, and flat OSM schema and the annotation sparsity. This paper tackles the alignment of OSM tags with the corresponding knowledge graph classes holistically by jointly considering the schema and instance layers. We propose a novel neural architecture that capitalizes upon a shared latent space for tag-to-class alignment created using linked entities in OSM and knowledge graphs. Our experiments performed to align OSM datasets for several countries with two of the most prominent openly available knowledge graphs, namely, Wikidata and DBpedia, demonstrate that the proposed approach outperforms the state-of-the-art schema alignment baselines by up to 53 percentage points in terms of F1-score. The resulting alignment facilitates new semantic annotations for over 10 million OSM entities worldwide, which is more than a 400% increase compared to the existing semantic annotations in OSM.
Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth
Venue: arXiv cs.CV 2021
Year: 2021

Current geometry-based monocular 3D object detection models can efficiently detect objects by leveraging perspective geometry, but their performance is limited due to the absence of accurate depth information. Though this issue can be alleviated in a depth-based model where a depth estimation module is plugged to predict depth information before 3D box reasoning, the introduction of such module dramatically reduces the detection speed. Instead of training a costly depth estimator, we propose a rendering module to augment the training data by synthesizing images with virtual-depths. The rendering module takes as input the RGB image and its corresponding sparse depth image, outputs a variety of photo-realistic synthetic images, from which the detection model can learn more discriminative features to adapt to the depth changes of the objects. Besides, we introduce an auxiliary module to improve the detection model by jointly optimizing it through a depth estimation task. Both modules are working in the training time and no extra computation will be introduced to the detection model. Experiments show that by working with our proposed modules, a geometry-based model can represent the leading accuracy on the KITTI 3D detection benchmark.
C3O: Collaborative Cluster Configuration Optimization for Distributed Data Processing in Public Clouds
Venue: arXiv cs.DC 2021
Year: 2021

Distributed dataflow systems enable data-parallel processing of large datasets on clusters. Public cloud providers offer a large variety and quantity of resources that can be used for such clusters. Yet, selecting appropriate cloud resources for dataflow jobs - that neither lead to bottlenecks nor to low resource utilization - is often challenging, even for expert users such as data engineers. We present C3O, a collaborative system for optimizing data processing cluster configurations in public clouds based on shared historical runtime data. The shared data is utilized for predicting the runtimes of data processing jobs on different possible cluster configurations, using specialized regression models. These models take the diverse execution contexts of different users into account and exhibit mean absolute errors below 3% in our experimental evaluation with 930 unique Spark jobs.
On the Approximability of Multistage Min-Sum Set Cover
Venue: arXiv cs.DS 2021
Year: 2021

We investigate the polynomial-time approximability of the multistage version of Min-Sum Set Cover ($\mathrm{DSSC}$), a natural and intriguing generalization of the classical List Update problem. In $\mathrm{DSSC}$, we maintain a sequence of permutations $(π^0, π^1, \ldots, π^T)$ on $n$ elements, based on a sequence of requests $(R^1, \ldots, R^T)$. We aim to minimize the total cost of updating $π^{t-1}$ to $π^{t}$, quantified by the Kendall tau distance $\mathrm{D}_{\mathrm{KT}}(π^{t-1}, π^t)$, plus the total cost of covering each request $R^t$ with the current permutation $π^t$, quantified by the position of the first element of $R^t$ in $π^t$. Using a reduction from Set Cover, we show that $\mathrm{DSSC}$ does not admit an $O(1)$-approximation, unless $\mathrm{P} = \mathrm{NP}$, and that any $o(\log n)$ (resp. $o(r)$) approximation to $\mathrm{DSSC}$ implies a sublogarithmic (resp. $o(r)$) approximation to Set Cover (resp. where each element appears at most $r$ times). Our main technical contribution is to show that $\mathrm{DSSC}$ can be approximated in polynomial-time within a factor of $O(\log^2 n)$ in general instances, by randomized rounding, and within a factor of $O(r^2)$, if all requests have cardinality at most $r$, by deterministic rounding.
Jarvis for Aeroengine Analytics: A Speech Enhanced Virtual Reality Demonstrator Based on Mining Knowledge Databases
Venue: arXiv cs.HC 2021
Year: 2021

In this paper, we present a Virtual Reality (VR) based environment where the engineer interacts with incoming data from a fleet of aeroengines. This data takes the form of 3D computer-aided design (CAD) engine models coupled with characteristic plots for the subsystems of each engine. Both the plots and models can be interacted with and manipulated using speech or gestural input. The characteristic data is ported to a knowledge-based system underpinned by a knowledge-graph storing complex domain knowledge. This permits the system to respond to queries about the current state and health of each aeroengine asset. Responses to these questions require some degree of analysis, which is handled by a semantic knowledge representation layer managing information on aeroengine subsystems. This paper represents a significant step forward for aeroengine analysis in a bespoke VR environment and brings us a step closer to a Jarvis-like system for aeroengine analytics.
Predicting the Future from First Person (Egocentric) Vision: A Survey
Venue: arXiv cs.CV 2021
Year: 2021

Egocentric videos can bring a lot of information about how humans perceive the world and interact with the environment, which can be beneficial for the analysis of human behaviour. The research in egocentric video analysis is developing rapidly thanks to the increasing availability of wearable devices and the opportunities offered by new large-scale egocentric datasets. As computer vision techniques continue to develop at an increasing pace, the tasks related to the prediction of future are starting to evolve from the need of understanding the present. Predicting future human activities, trajectories and interactions with objects is crucial in applications such as human-robot interaction, assistive wearable technologies for both industrial and daily living scenarios, entertainment and virtual or augmented reality. This survey summarises the evolution of studies in the context of future prediction from egocentric vision making an overview of applications, devices, existing problems, commonly used datasets, models and input modalities. Our analysis highlights that methods for future prediction from egocentric vision can have a significant impact in a range of applications and that further research efforts should be devoted to the standardisation of tasks and the proposal of datasets considering real-world scenarios such as the ones with an industrial vocation.
On Optimal Quantization in Sequential Detection
Venue: arXiv cs.IT 2021
Year: 2021

The problem of designing optimal quantization rules for sequential detectors is investigated. First, it is shown that this task can be solved within the general framework of active sequential detection. Using this approach, the optimal sequential detector and the corresponding quantizer are characterized and their properties are briefly discussed. In particular, it is shown that designing optimal quantization rules requires solving a nonconvex optimization problem, which can lead to issues in terms of computational complexity and numerical stability. Motivated by these difficulties, two performance bounds are proposed that are easier to evaluate than the true performance measures and are potentially tighter than the bounds currently available in the literature. The usefulness of the bounds and the properties of the optimal quantization rules are illustrated with two numerical examples.
A Signal Detection Scheme Based on Deep Learning in OFDM Systems
Venue: arXiv cs.IT 2021
Year: 2021

Channel estimation and signal detection are essential steps to ensure the quality of end-to-end communication in orthogonal frequency-division multiplexing (OFDM) systems. In this paper, we develop a DDLSD approach, i.e., Data-driven Deep Learning for Signal Detection in OFDM systems. First, the OFDM system model is established. Then, the long short-term memory (LSTM) is introduced into the OFDM system model. Wireless channel data is generated through simulation, the preprocessed time series feature information is input into the LSTM to complete the offline training. Finally, the trained model is used for online recovery of transmitted signal. The difference between this scheme and existing OFDM receiver is that explicit estimated channel state information (CSI) is transformed into invisible estimated CSI, and the transmit symbol is directly restored. Simulation results show that the DDLSD scheme outperforms the existing traditional methods in terms of improving channel estimation and signal detection performance.
Functorial String Diagrams for Reverse-Mode Automatic Differentiation
Venue: arXiv cs.PL 2021
Year: 2021

We enhance the calculus of string diagrams for monoidal categories with hierarchical features in order to capture closed monoidal (and cartesian closed) structure. Using this new syntax we formulate an automatic differentiation algorithm for (applied) simply typed lambda calculus in the style of [Pearlmutter and Siskind 2008] and we prove for the first time its soundness. To give an efficient yet principled implementation of the AD algorithm we define a sound and complete representation of hierarchical string diagrams as a class of hierarchical hypergraphs we call hypernets.
Surrogate Model-Based Explainability Methods for Point Cloud NNs
Venue: arXiv cs.CV 2021
Year: 2021

In the field of autonomous driving and robotics, point clouds are showing their excellent real-time performance as raw data from most of the mainstream 3D sensors. Therefore, point cloud neural networks have become a popular research direction in recent years. So far, however, there has been little discussion about the explainability of deep neural networks for point clouds. In this paper, we propose a point cloud-applicable explainability approach based on local surrogate model-based method to show which components contribute to the classification. Moreover, we propose quantitative fidelity validations for generated explanations that enhance the persuasive power of explainability and compare the plausibility of different existing point cloud-applicable explainability methods. Our new explainability approach provides a fairly accurate, more semantically coherent and widely applicable explanation for point cloud classification tasks. Our code is available at https://github.com/Explain3D/LIME-3D
A family of codes with variable locality and availability
Venue: arXiv cs.IT 2021
Year: 2021

In this work we present a class of locally recoverable codes, i.e. codes where an erasure at a position $P$ of a codeword may be recovered from the knowledge of the entries in the positions of a recovery set $R_P$. The codes in the class that we define have availability, meaning that for each position $P$ there are several distinct recovery sets. Also, the entry at position $P$ may be recovered even in the presence of erasures in some of the positions of the recovery sets, and the number of supported erasures may vary among the various recovery sets.
Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNs
Venue: arXiv cs.LG 2021
Year: 2021

Quantization is a technique for reducing deep neural networks (DNNs) training and inference times, which is crucial for training in resource constrained environments or applications where inference is time critical. State-of-the-art (SOTA) quantization approaches focus on post-training quantization, i.e., quantization of pre-trained DNNs for speeding up inference. While work on quantized training exists, most approaches require refinement in full precision (usually single precision) in the final training phase or enforce a global word length across the entire DNN. This leads to suboptimal assignments of bit-widths to layers and, consequently, suboptimal resource usage. In an attempt to overcome such limitations, we introduce AdaPT, a new fixed-point quantized sparsifying training strategy. AdaPT decides about precision switches between training epochs based on information theoretic conditions. The goal is to determine on a per-layer basis the lowest precision that causes no quantization-induced information loss while keeping the precision high enough such that future learning steps do not suffer from vanishing gradients. The benefits of the resulting fully quantized DNN are evaluated based on an analytical performance model which we develop. We illustrate that an average speedup of 1.27 compared to standard training in float32 with an average accuracy increase of 0.98% can be achieved for AlexNet/ResNet on CIFAR10/100 and we further demonstrate these AdaPT trained models achieve an average inference speedup of 2.33 with a model size reduction of 0.52.
Models of Computational Profiles to Study the Likelihood of DNN Metamorphic Test Cases
Venue: arXiv cs.LG 2021
Year: 2021

Neural network test cases are meant to exercise different reasoning paths in an architecture and used to validate the prediction outcomes. In this paper, we introduce "computational profiles" as vectors of neuron activation levels. We investigate the distribution of computational profile likelihood of metamorphic test cases with respect to the likelihood distributions of training, test and error control cases. We estimate the non-parametric probability densities of neuron activation levels for each distinct output class. Probabilities are inferred using training cases only, without any additional knowledge about metamorphic test cases. Experiments are performed by training a network on the MNIST Fashion library of images and comparing prediction likelihoods with those obtained from error control-data and from metamorphic test cases. Experimental results show that the distributions of computational profile likelihood for training and test cases are somehow similar, while the distribution of the random-noise control-data is always remarkably lower than the observed one for the training and testing sets. In contrast, metamorphic test cases show a prediction likelihood that lies in an extended range with respect to training, tests, and random noise. Moreover, the presented approach allows the independent assessment of different training classes and experiments to show that some of the classes are more sensitive to misclassifying metamorphic test cases than other classes. In conclusion, metamorphic test cases represent very aggressive tests for neural network architectures. Furthermore, since metamorphic test cases force a network to misclassify those inputs whose likelihood is similar to that of training cases, they could also be considered as adversarial attacks that evade defenses based on computational profile likelihood evaluation.
Toward Integrated Human-machine Intelligence for Civil Engineering: An Interdisciplinary Perspective
Venue: arXiv cs.AI 2021
Year: 2021

The purpose of this paper is to examine the opportunities and barriers of Integrated Human-Machine Intelligence (IHMI) in civil engineering. Integrating artificial intelligence's high efficiency and repeatability with humans' adaptability in various contexts can advance timely and reliable decision-making during civil engineering projects and emergencies. Successful cases in other domains, such as biomedical science, healthcare, and transportation, showed the potential of IHMI in data-driven, knowledge-based decision-making in numerous civil engineering applications. However, whether the industry and academia are ready to embrace the era of IHMI and maximize its benefit to the industry is still questionable due to several knowledge gaps. This paper thus calls for future studies in exploring the value, method, and challenges of applying IHMI in civil engineering. Our systematic review of the literature and motivating cases has identified four knowledge gaps in achieving effective IHMI in civil engineering. First, it is unknown what types of tasks in the civil engineering domain can be assisted by AI and to what extent. Second, the interface between human and AI in civil engineering-related tasks need more precise and formal definition. Third, the barriers that impede collecting detailed behavioral data from humans and contextual environments deserve systematic classification and prototyping. Lastly, it is unknown what expected and unexpected impacts will IHMI have on the AEC industry and entrepreneurship. Analyzing these knowledge gaps led to a list of identified research questions. This paper will lay the foundation for identifying relevant studies to form a research roadmap to address the four knowledge gaps identified.