OpenRaiser/Intern-Atlas huggingface dataset:
Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction
Venue: arXiv cs.CV 2021
Year: 2021

Event cameras are novel vision sensors that report per-pixel brightness changes as a stream of asynchronous "events". They offer significant advantages compared to standard cameras due to their high temporal resolution, high dynamic range and lack of motion blur. However, events only measure the varying component of the visual signal, which limits their ability to encode scene context. By contrast, standard cameras measure absolute intensity frames, which capture a much richer representation of the scene. Both sensors are thus complementary. However, due to the asynchronous nature of events, combining them with synchronous images remains challenging, especially for learning-based methods. This is because traditional recurrent neural networks (RNNs) are not designed for asynchronous and irregular data from additional sensors. To address this challenge, we introduce Recurrent Asynchronous Multimodal (RAM) networks, which generalize traditional RNNs to handle asynchronous and irregular data from multiple sensors. Inspired by traditional RNNs, RAM networks maintain a hidden state that is updated asynchronously and can be queried at any time to generate a prediction. We apply this novel architecture to monocular depth estimation with events and frames where we show an improvement over state-of-the-art methods by up to 30% in terms of mean absolute depth error. To enable further research on multimodal learning with events, we release EventScape, a new dataset with events, intensity frames, semantic labels, and depth maps recorded in the CARLA simulator.
HVAQ: A High-Resolution Vision-Based Air Quality Dataset
Venue: arXiv cs.CV 2021
Year: 2021

Air pollutants, such as particulate matter, negatively impact human health. Most existing pollution monitoring techniques use stationary sensors, which are typically sparsely deployed. However, real-world pollution distributions vary rapidly with position and the visual effects of air pollution can be used to estimate concentration, potentially at high spatial resolution. Accurate pollution monitoring requires either densely deployed conventional point sensors, at-a-distance vision-based pollution monitoring, or a combination of both. The main contribution of this paper is that to the best of our knowledge, it is the first publicly available, high temporal and spatial resolution air quality dataset containing simultaneous point sensor measurements and corresponding images. The dataset enables, for the first time, high spatial resolution evaluation of image-based air pollution estimation algorithms. It contains PM2.5, PM10, temperature, and humidity data. We evaluate several state-of-art vision-based PM concentration estimation algorithms on our dataset and quantify the increase in accuracy resulting from higher point sensor density and the use of images. It is our intent and belief that this dataset can enable advances by other research teams working on air quality estimation. Our dataset is available at https://github.com/implicitDeclaration/HVAQ-dataset/tree/master.
On Typical Hesitant Fuzzy Languages and Automata
Venue: arXiv cs.FL 2021
Year: 2021

The idea of nondeterministic typical hesitant fuzzy automata is a generalization of the fuzzy automata presented by Costa and Bedregal. This paper, presents the sufficient and necessary conditions for a typical hesitant fuzzy language to be computed by nondeterministic typical hesitant fuzzy automata. Besides, the paper introduces a new class of Typical Hesitant Fuzzy Automata with crisp transitions, and we will show that this new class is equivalent to the original class introduced by Costa and Bedregal
A Study on the Manifestation of Trust in Speech
Venue: arXiv cs.HC 2021
Year: 2021

Research has shown that trust is an essential aspect of human-computer interaction directly determining the degree to which the person is willing to use a system. An automatic prediction of the level of trust that a user has on a certain system could be used to attempt to correct potential distrust by having the system take relevant actions like, for example, apologizing or explaining its decisions. In this work, we explore the feasibility of automatically detecting the level of trust that a user has on a virtual assistant (VA) based on their speech. We developed a novel protocol for collecting speech data from subjects induced to have different degrees of trust in the skills of a VA. The protocol consists of an interactive session where the subject is asked to respond to a series of factual questions with the help of a virtual assistant. In order to induce subjects to either trust or distrust the VA's skills, they are first informed that the VA was previously rated by other users as being either good or bad; subsequently, the VA answers the subjects' questions consistently to its alleged abilities. All interactions are speech-based, with subjects and VAs communicating verbally, which allows the recording of speech produced under different trust conditions. Using this protocol, we collected a speech corpus in Argentine Spanish. We show clear evidence that the protocol effectively succeeded in influencing subjects into the desired mental state of either trusting or distrusting the agent's skills, and present results of a perceptual study of the degree of trust performed by expert listeners. Finally, we found that the subject's speech can be used to detect which type of VA they were using, which could be considered a proxy for the user's trust toward the VA's abilities, with an accuracy up to 76%, compared to a random baseline of 50%.
A Reinforcement learning method for Optical Thin-Film Design
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning, especially deep learning, is dramatically changing the methods associated with optical thin-film inverse design. The vast majority of this research has focused on the parameter optimization (layer thickness, and structure size) of optical thin-films. A challenging problem that arises is an automated material search. In this work, we propose a new end-to-end algorithm for optical thin-film inverse design. This method combines the ability of unsupervised learning, reinforcement learning(RL) and includes a genetic algorithm to design an optical thin-film without any human intervention. Furthermore, with several concrete examples, we have shown how one can use this technique to optimize the spectra of a multi-layer solar absorber device.
Blockchain in agriculture
Venue: arXiv cs.CY 2021
Year: 2021

Blockchain is an emerging digital technology allowing ubiquitous financial transactions among distributed untrusted parties, without the need of intermediaries such as banks. This chapter examines the impact of blockchain technology in agriculture and food supply chain, presents existing ongoing projects and initiatives, and discusses overall implications, challenges and potential, with a critical view over the maturity of these projects. Our findings indicate that blockchain is a promising technology towards a transparent supply chain of food, with many ongoing initiatives in various food products and food-related issues, but many barriers and challenges still exist, which hinder its wider popularity among farmers and systems. These challenges involve technical aspects, education, policies and regulatory frameworks.
A Machine Learning model of the combination of normalized SD1 and SD2 indexes from 24h-Heart Rate Variability as a predictor of myocardial infarction
Venue: arXiv cs.LG 2021
Year: 2021

Aim: to evaluate the ability of the nonlinear 24-HRV as a predictor of MI using Machine Learning Methods: The sample was composed of 218 patients divided into two groups (Healthy, n=128; MI n=90). The sample dataset is part of the Telemetric and Holter Electrocardiogram Warehouse (THEW) database, from the University of Rochester Medical Center. We used the most common ML algorithms for accuracy comparison with a setting of 10-fold cross-validation (briefly, Linear Regression, Linear Discriminant Analysis, k-Nearest Neighbour, Random Forest, Supporting Vector Machine, Naïve Bayes, C 5.0 and Stochastic Gradient Boosting). Results: The main findings of this study show that the combination of SD1nu + SD2nu has greater predictive power for MI in comparison to other HRV indexes. Conclusion: The ML model using nonlinear HRV indexes showed to be more effective than the linear domain, evidenced through the application of ML, represented by a good precision of the Stochastic Gradient Boosting model. Keywords: heart rate variability, machine learning, nonlinear domain, cardiovascular disease
ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous CPSs are often required to handle uncertainties and self-manage the system operation in response to problems and increasing risk in the operating paradigm. This risk may arise due to distribution shifts, environmental context, or failure of software or hardware components. Traditional techniques for risk assessment focus on design-time techniques such as hazard analysis, risk reduction, and assurance cases among others. However, these static, design-time techniques do not consider the dynamic contexts and failures the systems face at runtime. We hypothesize that this requires a dynamic assurance approach that computes the likelihood of unsafe conditions or system failures considering the safety requirements, assumptions made at design time, past failures in a given operating context, and the likelihood of system component failures. We introduce the ReSonAte dynamic risk estimation framework for autonomous systems. ReSonAte reasons over Bow-Tie Diagrams (BTDs) which capture information about hazard propagation paths and control strategies. Our innovation is the extension of the BTD formalism with attributes for modeling the conditional relationships with the state of the system and environment. We also describe a technique for estimating these conditional relationships and equations for estimating risk based on the state of the system and environment. To help with this process, we provide a scenario modeling procedure that can use the prior distributions of the scenes and threat conditions to generate the data required for estimating the conditional relationships. To improve scalability and reduce the amount of data required, this process considers each control strategy in isolation and composes several single-variate distributions into one complete multi-variate distribution for the control strategy in question.
Dynamic VNF Placement, Resource Allocation and Traffic Routing in 5G
Venue: arXiv cs.NI 2021
Year: 2021

5G networks are going to support a variety of vertical services, with a diverse set of key performance indicators (KPIs), by using enabling technologies such as software-defined networking and network function virtualization. It is the responsibility of the network operator to efficiently allocate the available resources to the service requests in such a way to honor KPI requirements, while accounting for the limited quantity of available resources and their cost. A critical challenge is that requests may be highly varying over time, requiring a solution that accounts for their dynamic generation and termination. With this motivation, we seek to make joint decisions for request admission, resource activation, VNF placement, resource allocation, and traffic routing. We do so by considering real-world aspects such as the setup times of virtual machines, with the goal of maximizing the mobile network operator profit. To this end, first, we formulate a one-shot optimization problem which can attain the optimum solution for small size problems given the complete knowledge of arrival and departure times of requests over the entire system lifespan. We then propose an efficient and practical heuristic solution that only requires this knowledge for the next time period and works for realistically-sized scenarios. Finally, we evaluate the performance of these solutions using real-world services and large-scale network topologies. {Results demonstrate that our heuristic solution performs better than a state-of-the-art online approach and close to the optimum.
AggFT: Low-Cost Fault-Tolerant Smart Meter Aggregation with Proven Termination and Privacy
Venue: arXiv cs.CR 2021
Year: 2021

Smart meter data aggregation protocols have been developed to address rising privacy threats against customers' consumption data. However, these protocols do not work satisfactorily in the presence of failures of smart meters or network communication links. In this paper, we propose a lightweight and fault-tolerant aggregation algorithm that can serve as a solid foundation for further research. We revisit an existing error-resilient privacy-preserving aggregation protocol based on masking and improve it by: (i) performing changes in the cryptographic parts that lead to a reduction of computational costs, (ii) simplifying the behaviour of the protocol in the presence of faults, and showing a proof of proper termination under a well-defined failure model, (iii) decoupling the computation part from the data flow so that the algorithm can also be used with homomorphic encryption as a basis for privacy-preservation. To best of our knowledge, this is the first algorithm that is formulated for both, masking and homomorphic encryption. (iv) Finally, we provide a formal proof of the privacy guarantee under failure. The systematic treatment with strict proofs and the established connection to graph theory may also serve as a starting point for possible generalizations and improvements with respect to increased resilience.
Security audit logging in microservice-based systems: survey of architecture patterns
Venue: arXiv cs.CR 2021
Year: 2021

Objective. Service-oriented architecture increases technical abilities for attacker to move laterally and maintain multiple pivot points inside of compromised environment. Microservice-based infrastructure brings more challenges for security architect related to internal event visibility and monitoring. Properly implemented logging and audit approach is a baseline for security operations and incident management. The aim of this study is to provide helpful resource to application and product security architects, software and operation engineers on existing architecture patterns to implement trustworthy logging and audit process in microservice-based environments. Method. In this paper, we conduct information security threats modeling and a systematic review of major electronic databases and libraries, security standards and presentations at the major security conferences as well as architecture whitepapers of industry vendors with relevant products. Results and practical relevance. In this work based on research papers and major security conferences presentations analysis, we identified industry best practices in logging audit patterns and its applicability depending on environment characteristic. We provided threat modeling for typical architecture pattern of logging system and identified 8 information security threats. We provided security threat mitigation and as a result of 11 high-level security requirements for audit logging system were identified. High-level security requirements can be used by application security architect in order to secure their products.
Range Minimum Queries in Minimal Space
Venue: arXiv cs.DS 2021
Year: 2021

We consider the problem of computing a sequence of range minimum queries. We assume a sequence of commands that contains values and queries. Our goal is to quickly determine the minimum value that exists between the current position and a previous position $i$. Range minimum queries are used as a sub-routine of several algorithms, namely related to string processing. We propose a data structure that can process these commands sequences. We obtain efficient results for several variations of the problem, in particular we obtain $O(1)$ time per command for the offline version and $O(α(n))$ amortized time for the online version, where $α(n)$ is the inverse Ackermann function and $n$ the number of values in the sequence. This data structure also has very small space requirements, namely $O(\ell)$ where $\ell$ is the maximum number active $i$ positions. We implemented our data structure and show that it is competitive against existing alternatives. We obtain comparable command processing time, in the nano second range, and much smaller space requirements.
iX-BSP: Incremental Belief Space Planning
Venue: arXiv cs.RO 2021
Year: 2021

Deciding what's next? is a fundamental problem in robotics and Artificial Intelligence. Under belief space planning (BSP), in a partially observable setting, it involves calculating the expected accumulated belief-dependent reward, where the expectation is with respect to all future measurements. Since solving this general un-approximated problem quickly becomes intractable, state of the art approaches turn to approximations while still calculating planning sessions from scratch. In this work we propose a novel paradigm, Incremental BSP (iX-BSP), based on the key insight that calculations across planning sessions are similar in nature and can be appropriately re-used. We calculate the expectation incrementally by utilizing Multiple Importance Sampling techniques for selective re-sampling and re-use of measurement from previous planning sessions. The formulation of our approach considers general distributions and accounts for data association aspects. We demonstrate how iX-BSP could benefit existing approximations of the general problem, introducing iML-BSP, which re-uses calculations across planning sessions under the common Maximum Likelihood assumption. We evaluate both methods and demonstrate a substantial reduction in computation time while statistically preserving accuracy. The evaluation includes both simulation and real-world experiments considering autonomous vision-based navigation and SLAM. As a further contribution, we introduce to iX-BSP the non-integral wildfire approximation, allowing one to trade accuracy for computational performance by averting from updating re-used beliefs when they are "close enough". We evaluate iX-BSP under wildfire demonstrating a substantial reduction in computation time while controlling the accuracy sacrifice. We also provide analytical and empirical bounds of the effect wildfire holds over the objective value.
A Latent Space Model for Multilayer Network Data
Venue: arXiv cs.SI 2021
Year: 2021

In this work, we propose a Bayesian statistical model to simultaneously characterize two or more social networks defined over a common set of actors. The key feature of the model is a hierarchical prior distribution that allows us to represent the entire system jointly, achieving a compromise between dependent and independent networks. Among others things, such a specification easily allows us to visualize multilayer network data in a low-dimensional Euclidean space, generate a weighted network that reflects the consensus affinity between actors, establish a measure of correlation between networks, assess cognitive judgements that subjects form about the relationships among actors, and perform clustering tasks at different social instances. Our model's capabilities are illustrated using several real-world data sets, taking into account different types of actors, sizes, and relations.
Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation
Venue: arXiv cs.ET 2021
Year: 2021

Convolution neural network (CNN), as one of the most powerful and popular technologies, has achieved remarkable progress for image and video classification since its invention in 1989. However, with the high definition video-data explosion, convolution layers in the CNN architecture will occupy a great amount of computing time and memory resources due to high computation complexity of matrix multiply accumulate operation. In this paper, a novel integrated photonic CNN is proposed based on double correlation operations through interleaved time-wavelength modulation. Micro-ring based multi-wavelength manipulation and single dispersion medium are utilized to realize convolution operation and replace the conventional optical delay lines. 200 images are tested in MNIST datasets with accuracy of 85.5% in our photonic CNN versus 86.5% in 64-bit computer.We also analyze the computing error of photonic CNN caused by various micro-ring parameters, operation baud rates and the characteristics of micro-ring weighting bank. Furthermore, a tensor processing unit based on 4x4 mesh with 1.2 TOPS (operation per second when 100% utilization) computing capability at 20G baud rate is proposed and analyzed to form a paralleled photonic CNN.
Using Jupyter for reproducible scientific workflows
Venue: arXiv cs.MS 2021
Year: 2021

Literate computing has emerged as an important tool for computational studies and open science, with growing folklore of best practices. In this work, we report two case studies - one in computational magnetism and another in computational mathematics - where domain-specific software was exposed to the Jupyter environment. This enables high-level control of simulations and computation, interactive exploration of computational results, batch processing on HPC resources, and reproducible workflow documentation in Jupyter notebooks. In the first study, Ubermag drives existing computational micromagnetics software through a domain-specific language embedded in Python. In the second study, a dedicated Jupyter kernel interfaces with the GAP system for computational discrete algebra and its dedicated programming language. In light of these case studies, we discuss the benefits of this approach, including progress toward more reproducible and reusable research results and outputs, notably through the use of infrastructure such as JupyterHub and Binder.
Benefits of Linear Conditioning with Metadata for Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Medical images are often accompanied by metadata describing the image (vendor, acquisition parameters) and the patient (disease type or severity, demographics, genomics). This metadata is usually disregarded by image segmentation methods. In this work, we adapt a linear conditioning method called FiLM (Feature-wise Linear Modulation) for image segmentation tasks. This FiLM adaptation enables integrating metadata into segmentation models for better performance. We observed an average Dice score increase of 5.1% on spinal cord tumor segmentation when incorporating the tumor type with FiLM. The metadata modulates the segmentation process through low-cost affine transformations applied on feature maps which can be included in any neural network's architecture. Additionally, we assess the relevance of segmentation FiLM layers for tackling common challenges in medical imaging: multi-class training with missing segmentations, model adaptation to multiple tasks, and training with a limited or unbalanced number of annotated data. Our results demonstrated the following benefits of FiLM for segmentation: FiLMed U-Net was robust to missing labels and reached higher Dice scores with few labels (up to 16.7%) compared to single-task U-Net. The code is open-source and available at www.ivadomed.org.