OpenRaiser/Intern-Atlas huggingface dataset:
Audience Response Prediction from Textual Context
Venue: arXiv cs.HC 2022
Year: 2022

Humans' perception system closely monitors audio-visual cues during multiparty interactions to react timely and naturally. Learning to predict timing and type of reaction responses during human-human interactions may help us to enrich human-computer interaction applications. In this paper we consider a presenter-audience setting and define an audience response prediction task from the presenter's textual speech. The task is formulated as a binary classification problem as occurrence and absence of response after the presenter's textual speech. We use the BERT model as our classifier and investigate models with different textual contexts under causal and non-causal prediction settings. While the non-causal textual context, one sentence preceding and one sentence following the response event, can hugely improve the accuracy of predictions, we showed that longer textual contexts with causal settings attain UAR and $F1$-Score improvements matching and exceeding the non-causal textual context performance within the experimental evaluations on the OPUS and TED datasets.
Single Morphing Attack Detection using Siamese Network and Few-shot Learning
Venue: arXiv cs.CV 2022
Year: 2022

Face morphing attack detection is challenging and presents a concrete and severe threat for face verification systems. Reliable detection mechanisms for such attacks, which have been tested with a robust cross-database protocol and unknown morphing tools still is a research challenge. This paper proposes a framework following the Few-Shot-Learning approach that shares image information based on the siamese network using triplet-semi-hard-loss to tackle the morphing attack detection and boost the clustering classification process. This network compares a bona fide or potentially morphed image with triplets of morphing and bona fide face images. Our results show that this new network cluster the data points, and assigns them to classes in order to obtain a lower equal error rate in a cross-database scenario sharing only small image numbers from an unknown database. Few-shot learning helps to boost the learning process. Experimental results using a cross-datasets trained with FRGCv2 and tested with FERET and the AMSL open-access databases reduced the BPCER10 from 43% to 4.91% using ResNet50 and 5.50% for MobileNetV2.
AdvSmo: Black-box Adversarial Attack by Smoothing Linear Structure of Texture
Venue: arXiv cs.CV 2022
Year: 2022

Black-box attacks usually face two problems: poor transferability and the inability to evade the adversarial defense. To overcome these shortcomings, we create an original approach to generate adversarial examples by smoothing the linear structure of the texture in the benign image, called AdvSmo. We construct the adversarial examples without relying on any internal information to the target model and design the imperceptible-high attack success rate constraint to guide the Gabor filter to select appropriate angles and scales to smooth the linear texture from the input images to generate adversarial examples. Benefiting from the above design concept, AdvSmo will generate adversarial examples with strong transferability and solid evasiveness. Finally, compared to the four advanced black-box adversarial attack methods, for the eight target models, the results show that AdvSmo improves the average attack success rate by 9% on the CIFAR-10 and 16% on the Tiny-ImageNet dataset compared to the best of these attack methods.
Monte Carlo Methods for Industry 4.0 Applications
Venue: arXiv cs.IT 2022
Year: 2022

The fourth industrial revolution and the digital transformation, commonly known as Industry 4.0, is exponentially progressing in recent years. Connected computers, devices, and intelligent machines communicate with each other and interact with the environment to require only a minimum of human intervention. An important issue in Industry 4.0 is the evaluation of the quality of the process in terms of KPIs. Monte Carlo simulations can play an important role to improve the estimations. However, there is still a lack of clear workflow to conduct the Monte Carlo simulations for selecting different Monte Carlo methods. This paper, therefore, proposes a simulation flow for conducting Monte Carlo methods comparison in Industry 4.0 applications. Based on the simulation flow, we compare Cumulative Monte Carlo and Markov Chain Monte Carlo methods. The experimental results show the way to use the Monte Carlo methods in Industry 4.0 and possible limitations of the two simulation methods.
World of Bugs: A Platform for Automated Bug Detection in 3D Video Games
Venue: arXiv cs.SE 2022
Year: 2022

We present World of Bugs (WOB), an open platform that aims to support Automated Bug Detection (ABD) research in video games. We discuss some open problems in ABD and how they relate to the platform's design, arguing that learning-based solutions are required if further progress is to be made. The platform's key feature is a growing collection of common video game bugs that may be used for training and evaluating ABD approaches.
Functional or imperative? On pleasant semantics for differentiable programming languages
Venue: arXiv cs.PL 2022
Year: 2022

In machine learning (ML), researchers and engineers seem to be at odds. System implementers would prefer models to be declarative, with detailed type information and semantic restrictions that allow models to be optimised, rearranged and parallelised. Yet practitioners show an overwhelming preference for dynamic, imperative languages with mutable state, and much engineering effort is spent bridging the resulting semantic divide. Is there a fundamental conflict? This article explores why imperative and functional styles are used, and how future language designs might get the best of both worlds.
Understanding the Properties of Generated Corpora
Venue: arXiv cs.CL 2022
Year: 2022

Models for text generation have become focal for many research tasks and especially for the generation of sentence corpora. However, understanding the properties of an automatically generated text corpus remains challenging. We propose a set of tools that examine the properties of generated text corpora. Applying these tools on various generated corpora allowed us to gain new insights into the properties of the generative models. As part of our characterization process, we found remarkable differences in the corpora generated by two leading generative technologies.
Reservoir Computing in robotics: a review
Venue: arXiv cs.RO 2022
Year: 2022

Reservoir Computing is a relatively new framework created to allow the usage of powerful but complex systems as computational mediums. The basic approach consists in training only a readout layer, exploiting the innate separation and transformation provided by the previous, untrained system. This approach has shown to possess great computational capabilities and is successfully used to achieve many tasks. This review aims to represent the current 'state-of-the-art' of the usage of Reservoir Computing techniques in the robotic field. An introductory description of the framework and its implementations is initially given. Subsequently, a summary of interesting applications, approaches, and solutions is presented and discussed. Considerations, ideas and possible future developments are proposed in the explanation.
Recommendations for Systematic Research on Emergent Language
Venue: arXiv cs.MA 2022
Year: 2022

Emergent language is unique among fields within the discipline of machine learning for its open-endedness, not obviously presenting well-defined problems to be solved. As a result, the current research in the field has largely been exploratory: focusing on establishing new problems, techniques, and phenomena. Yet after these problems have been established, subsequent progress requires research which can measurably demonstrate how it improves on prior approaches. This type of research is what we call systematic research; in this paper, we illustrate this mode of research specifically for emergent language. We first identify the overarching goals of emergent language research, categorizing them as either science or engineering. Using this distinction, we present core methodological elements of science and engineering, analyze their role in current emergent language research, and recommend how to apply these elements.
Synthetic Data-Based Simulators for Recommender Systems: A Survey
Venue: arXiv cs.IR 2022
Year: 2022

This survey aims at providing a comprehensive overview of the recent trends in the field of modeling and simulation (M&S) of interactions between users and recommender systems and applications of the M&S to the performance improvement of industrial recommender engines. We start with the motivation behind the development of frameworks implementing the simulations -- simulators -- and the usage of them for training and testing recommender systems of different types (including Reinforcement Learning ones). Furthermore, we provide a new consistent classification of existing simulators based on their functionality, approbation, and industrial effectiveness and moreover make a summary of the simulators found in the research literature. Besides other things, we discuss the building blocks of simulators: methods for synthetic data (user, item, user-item responses) generation, methods for what-if experimental analysis, methods and datasets used for simulation quality evaluation (including the methods that monitor and/or close possible simulation-to-reality gaps), and methods for summarization of experimental simulation results. Finally, this survey considers emerging topics and open problems in the field.
Doubly Reparameterized Importance Weighted Structure Learning for Scene Graph Generation
Venue: arXiv cs.CV 2022
Year: 2022

As a structured prediction task, scene graph generation, given an input image, aims to explicitly model objects and their relationships by constructing a visually-grounded scene graph. In the current literature, such task is universally solved via a message passing neural network based mean field variational Bayesian methodology. The classical loose evidence lower bound is generally chosen as the variational inference objective, which could induce oversimplified variational approximation and thus underestimate the underlying complex posterior. In this paper, we propose a novel doubly reparameterized importance weighted structure learning method, which employs a tighter importance weighted lower bound as the variational inference objective. It is computed from multiple samples drawn from a reparameterizable Gumbel-Softmax sampler and the resulting constrained variational inference task is solved by a generic entropic mirror descent algorithm. The resulting doubly reparameterized gradient estimator reduces the variance of the corresponding derivatives with a beneficial impact on learning. The proposed method achieves the state-of-the-art performance on various popular scene graph generation benchmarks.
Monocular Spherical Depth Estimation with Explicitly Connected Weak Layout Cues
Venue: arXiv cs.CV 2022
Year: 2022

Spherical cameras capture scenes in a holistic manner and have been used for room layout estimation. Recently, with the availability of appropriate datasets, there has also been progress in depth estimation from a single omnidirectional image. While these two tasks are complementary, few works have been able to explore them in parallel to advance indoor geometric perception, and those that have done so either relied on synthetic data, or used small scale datasets, as few options are available that include both layout annotations and dense depth maps in real scenes. This is partly due to the necessity of manual annotations for room layouts. In this work, we move beyond this limitation and generate a 360 geometric vision (360V) dataset that includes multiple modalities, multi-view stereo data and automatically generated weak layout cues. We also explore an explicit coupling between the two tasks to integrate them into a singleshot trained model. We rely on depth-based layout reconstruction and layout-based depth attention, demonstrating increased performance across both tasks. By using single 360 cameras to scan rooms, the opportunity for facile and quick building-scale 3D scanning arises.
SensorStream: An XES Extension for Enriching Event Logs with IoT-Sensor Data
Venue: arXiv cs.SE 2022
Year: 2022

Process management and process orchestration/execution are currently hot topics; prevalent trends such as automation and Industry 4.0 require solutions which allow domain-experts to easily model and execute processes in various domains, including manufacturing and health-care. These domains, in turn, rely on a tight integration between hardware and software, i.e. via the Internet of Things (IoT). While process execution is about actuation, i.e. actively triggering actions and awaiting their completion, accompanying IoT sensors monitor humans and the environment. These sensors produce large amounts of procedural, discrete, and continuous data streams, that hold the key to understanding the quality of process subjects (e.g. produced parts), outcome (e.g. quantity and quality), and error causes. Processes constantly evolve in conjunction with their IoT environment. This requires joint storage of data generated by processes, with data generated by the IoT sensors is therefore needed. In this paper, we present an extension of the process log standard format XES, namely SensorStream. SensorStream enables to connect IoT data to process events, as well as a set of semantic annotations to describe the scenario and environment during data collection. This allows to preserve the full context required for data-analysis, so that logs can be analyzed even when scenarios or hardware artifacts are rapidly changing. Through additional semantic annotations, we envision the XES extension log format to be a solid based for the creation of a (semi-)automatic analysis pipeline, which can support domain experts by automatically providing data visualization, or even process insights.
Linear recurrent cryptography: golden-like cryptography for higher order linear recurrences
Venue: arXiv cs.CR 2022
Year: 2022

We develop matrix cryptography based on linear recurrent sequences of any order that allows securing encryption against brute force and chosen plaintext attacks. In particular, we solve the problem of generalizing error detection and correction algorithms of golden cryptography previously known only for recurrences of a special form. They are based on proving the checking relations (inequalities satisfied by the ciphertext) under the condition that the analog of the golden $Q$-matrix has the strong Perron-Frobenius property. These algorithms are proved to be especially efficient when the characteristic polynomial of the recurrence is a Pisot polynomial. Finally, we outline algorithms for generating recurrences that satisfy our conditions.
Reflections in an octagonal mirror maze
Venue: arXiv cs.CG 2022
Year: 2022

Suppose we are given an environment consisting of axis-parallel and diagonal line segments with integer endpoints, each of which may be reflective or non-reflective, with integer endpoints, and an initial position for a light ray passing through points of the integer grid. Then in time polynomial in the number of segments and in the number of bits needed to specify the coordinates of the input, we can determine the eventual fate of the reflected ray.
LidarMultiNet: Unifying LiDAR Semantic Segmentation, 3D Object Detection, and Panoptic Segmentation in a Single Multi-task Network
Venue: arXiv cs.CV 2022
Year: 2022

This technical report presents the 1st place winning solution for the Waymo Open Dataset 3D semantic segmentation challenge 2022. Our network, termed LidarMultiNet, unifies the major LiDAR perception tasks such as 3D semantic segmentation, object detection, and panoptic segmentation in a single framework. At the core of LidarMultiNet is a strong 3D voxel-based encoder-decoder network with a novel Global Context Pooling (GCP) module extracting global contextual features from a LiDAR frame to complement its local features. An optional second stage is proposed to refine the first-stage segmentation or generate accurate panoptic segmentation results. Our solution achieves a mIoU of 71.13 and is the best for most of the 22 classes on the Waymo 3D semantic segmentation test set, outperforming all the other 3D semantic segmentation methods on the official leaderboard. We demonstrate for the first time that major LiDAR perception tasks can be unified in a single strong network that can be trained end-to-end.
Image-based Stability Quantification
Venue: arXiv cs.CV 2022
Year: 2022

Quantitative evaluation of human stability using foot pressure/force measurement hardware and motion capture (mocap) technology is expensive, time consuming, and restricted to the laboratory. We propose a novel image-based method to estimate three key components for stability computation: Center of Mass (CoM), Base of Support (BoS), and Center of Pressure (CoP). Furthermore, we quantitatively validate our image-based methods for computing two classic stability measures, CoMtoCoP and CoMtoBoS distances, against values generated directly from laboratory-based sensor output (ground truth) using a publicly available, multi-modality (mocap, foot pressure, two-view videos), ten-subject human motion dataset. Using Leave One Subject Out (LOSO) cross-validation, experimental results show: 1) our image-based CoM estimation method (CoMNet) consistently outperforms state-of-the-art inertial sensor-based CoM estimation techniques; 2) stability computed by our image-based method combined with insole foot pressure sensor data produces consistent, strong, and statistically significant correlation with ground truth stability measures (CoMtoCoP r = 0.79 p < 0.001, CoMtoBoS r = 0.75 p < 0.001); 3) our fully image-based estimation of stability produces consistent, positive, and statistically significant correlation on the two stability metrics (CoMtoCoP r = 0.31 p < 0.001, CoMtoBoS r = 0.22 p < 0.043). Our study provides promising quantitative evidence for the feasibility of image-based stability evaluation in natural environments.
Efficient Adaptive Federated Optimization of Federated Learning for IoT
Venue: arXiv cs.LG 2022
Year: 2022

The proliferation of the Internet of Things (IoT) and widespread use of devices with sensing, computing, and communication capabilities have motivated intelligent applications empowered by artificial intelligence. The classical artificial intelligence algorithms require centralized data collection and processing which are challenging in realistic intelligent IoT applications due to growing data privacy concerns and distributed datasets. Federated Learning (FL) has emerged as a distributed privacy-preserving learning framework that enables IoT devices to train global model through sharing model parameters. However, inefficiency due to frequent parameters transmissions significantly reduce FL performance. Existing acceleration algorithms consist of two main type including local update considering trade-offs between communication and computation and parameter compression considering trade-offs between communication and precision. Jointly considering these two trade-offs and adaptively balancing their impacts on convergence have remained unresolved. To solve the problem, this paper proposes a novel efficient adaptive federated optimization (EAFO) algorithm to improve efficiency of FL, which minimizes the learning error via jointly considering two variables including local update and parameter compression and enables FL to adaptively adjust the two variables and balance trade-offs among computation, communication and precision. The experiment results illustrate that comparing with state-of-the-art algorithms, the proposed EAFO can achieve higher accuracies faster.
A Framework for Understanding Model Extraction Attack and Defense
Venue: arXiv cs.LG 2022
Year: 2022

The privacy of machine learning models has become a significant concern in many emerging Machine-Learning-as-a-Service applications, where prediction services based on well-trained models are offered to users via pay-per-query. The lack of a defense mechanism can impose a high risk on the privacy of the server's model since an adversary could efficiently steal the model by querying only a few `good' data points. The interplay between a server's defense and an adversary's attack inevitably leads to an arms race dilemma, as commonly seen in Adversarial Machine Learning. To study the fundamental tradeoffs between model utility from a benign user's view and privacy from an adversary's view, we develop new metrics to quantify such tradeoffs, analyze their theoretical properties, and develop an optimization problem to understand the optimal adversarial attack and defense strategies. The developed concepts and theory match the empirical findings on the `equilibrium' between privacy and utility. In terms of optimization, the key ingredient that enables our results is a unified representation of the attack-defense problem as a min-max bi-level problem. The developed results will be demonstrated by examples and experiments.
A Review of Published Machine Learning Natural Language Processing Applications for Protocolling Radiology Imaging
Venue: arXiv cs.CV 2022
Year: 2022

Machine learning (ML) is a subfield of Artificial intelligence (AI), and its applications in radiology are growing at an ever-accelerating rate. The most studied ML application is the automated interpretation of images. However, natural language processing (NLP), which can be combined with ML for text interpretation tasks, also has many potential applications in radiology. One such application is automation of radiology protocolling, which involves interpreting a clinical radiology referral and selecting the appropriate imaging technique. It is an essential task which ensures that the correct imaging is performed. However, the time that a radiologist must dedicate to protocolling could otherwise be spent reporting, communicating with referrers, or teaching. To date, there have been few publications in which ML models were developed that use clinical text to automate protocol selection. This article reviews the existing literature in this field. A systematic assessment of the published models is performed with reference to best practices suggested by machine learning convention. Progress towards implementing automated protocolling in a clinical setting is discussed.
Explanatory causal effects for model agnostic explanations
Venue: arXiv cs.LG 2022
Year: 2022

This paper studies the problem of estimating the contributions of features to the prediction of a specific instance by a machine learning model and the overall contribution of a feature to the model. The causal effect of a feature (variable) on the predicted outcome reflects the contribution of the feature to a prediction very well. A challenge is that most existing causal effects cannot be estimated from data without a known causal graph. In this paper, we define an explanatory causal effect based on a hypothetical ideal experiment. The definition brings several benefits to model agnostic explanations. First, explanations are transparent and have causal meanings. Second, the explanatory causal effect estimation can be data driven. Third, the causal effects provide both a local explanation for a specific prediction and a global explanation showing the overall importance of a feature in a predictive model. We further propose a method using individual and combined variables based on explanatory causal effects for explanations. We show the definition and the method work with experiments on some real-world data sets.