OpenRaiser/Intern-Atlas huggingface dataset:
Dynamic imaging and characterization of volatile aerosols in e-cigarette emissions using deep learning-based holographic microscopy
Venue: arXiv cs.LG 2021
Year: 2021

Various volatile aerosols have been associated with adverse health effects; however, characterization of these aerosols is challenging due to their dynamic nature. Here we present a method that directly measures the volatility of particulate matter (PM) using computational microscopy and deep learning. This method was applied to aerosols generated by electronic cigarettes (e-cigs), which vaporize a liquid mixture (e-liquid) that mainly consists of propylene glycol (PG), vegetable glycerin (VG), nicotine, and flavoring compounds. E-cig generated aerosols were recorded by a field-portable computational microscope, using an impaction-based air sampler. A lensless digital holographic microscope inside this mobile device continuously records the inline holograms of the collected particles. A deep learning-based algorithm is used to automatically reconstruct the microscopic images of e-cig generated particles from their holograms, and rapidly quantify their volatility. To evaluate the effects of e-liquid composition on aerosol dynamics, we measured the volatility of the particles generated by flavorless, nicotine-free e-liquids with various PG/VG volumetric ratios, revealing a negative correlation between the particles' volatility and the volumetric ratio of VG in the e-liquid. For a given PG/VG composition, the addition of nicotine dominated the evaporation dynamics of the e-cig aerosol and the aforementioned negative correlation was no longer observed. We also revealed that flavoring additives in e-liquids significantly decrease the volatility of e-cig aerosol. The presented holographic volatility measurement technique and the associated mobile device might provide new insights on the volatility of e-cig generated particles and can be applied to characterize various volatile PM.
Security and Machine Learning Adoption in IoT: A Preliminary Study of IoT Developer Discussions
Venue: arXiv cs.CR 2021
Year: 2021

Internet of Things (IoT) is defined as the connection between places and physical objects (i.e., things) over the internet/network via smart computing devices. Traditionally, we learn about the IoT ecosystem/problems by conducting surveys of IoT developers/practitioners. Another way to learn is by analyzing IoT developer discussions in popular online developer forums like Stack Overflow (SO). However, we are aware of no such studies that focused on IoT developers' security and ML-related discussions in SO. This paper offers the results of preliminary study of IoT developer discussions in SO. We find around 12% of sentences contain security discussions, while around 0.12% sentences contain ML- related discussions. We find that IoT developers discussing security issues frequently inquired about how the shared data can be stored, shared, and transferred securely across IoT devices and users. We also find that IoT developers are interested to adopt deep neural network-based ML models into their IoT devices, but they find it challenging to accommodate those into their resource-constrained IoT devices. Our findings offer implications for IoT vendors and researchers to develop and design novel techniques for improved security and ML adoption into IoT devices.
HLE-UPC at SemEval-2021 Task 5: Multi-Depth DistilBERT for Toxic Spans Detection
Venue: arXiv cs.CL 2021
Year: 2021

This paper presents our submission to SemEval-2021 Task 5: Toxic Spans Detection. The purpose of this task is to detect the spans that make a text toxic, which is a complex labour for several reasons. Firstly, because of the intrinsic subjectivity of toxicity, and secondly, due to toxicity not always coming from single words like insults or offends, but sometimes from whole expressions formed by words that may not be toxic individually. Following this idea of focusing on both single words and multi-word expressions, we study the impact of using a multi-depth DistilBERT model, which uses embeddings from different layers to estimate the final per-token toxicity. Our quantitative results show that using information from multiple depths boosts the performance of the model. Finally, we also analyze our best model qualitatively.
GDPR Compliant Blockchains-A Systematic Literature Review
Venue: arXiv cs.CR 2021
Year: 2021

Although blockchain-based digital services promise trust, accountability, and transparency, multiple paradoxes between blockchains and GDPR have been highlighted in the recent literature. Some of the recent literature also proposed possible solutions to these paradoxes. This article aims to conduct a systematic literature review on GDPR compliant blockchains and synthesize the findings. In particular, the goal was to identify 1) the GDPR articles that have been explored in prior literature; 2) the relevant research domains that have been explored, and 3) the research gaps. Our findings synthesized that the blockchains relevant GDPR articles can be categorized into six major groups, namely data deletion and modification (Article 16, 17, and 18), protection by design by default (Article 25), responsibilities of controllers and processors (Article 24, 26, and 28), consent management (Article 7), data processing principles and lawfulness (Article 5,6 and 12), and territorial scope (Article 3). We also found seven research domains where GDPR compliant blockchains have been discussed, which include IoT, financial data, healthcare, personal identity, online data, information governance, and smart city. From our analysis, we have identified a few key research gaps and present a future research direction.
Reservoir-Based Distributed Machine Learning for Edge Operation
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a novel design for in-situ training of machine learning algorithms built into smart sensors, and illustrate distributed training scenarios using radio frequency (RF) spectrum sensors. Current RF sensors at the Edge lack the computational resources to support practical, in-situ training for intelligent signal classification. We propose a solution using Deepdelay Loop Reservoir Computing (DLR), a processing architecture that supports machine learning algorithms on resource-constrained edge-devices by leveraging delayloop reservoir computing in combination with innovative hardware. DLR delivers reductions in form factor, hardware complexity and latency, compared to the State-ofthe- Art (SoA) neural nets. We demonstrate DLR for two applications: RF Specific Emitter Identification (SEI) and wireless protocol recognition. DLR enables mobile edge platforms to authenticate and then track emitters with fast SEI retraining. Once delay loops separate the data classes, traditionally complex, power-hungry classification models are no longer needed for the learning process. Yet, even with simple classifiers such as Ridge Regression (RR), the complexity grows at least quadratically with the input size. DLR with a RR classifier exceeds the SoA accuracy, while further reducing power consumption by leveraging the architecture of parallel (split) loops. To authenticate mobile devices across large regions, DLR can be trained in a distributed fashion with very little additional processing and a small communication cost, all while maintaining accuracy. We illustrate how to merge locally trained DLR classifiers in use cases of interest.
Canonical and Surface Morphological Segmentation for Nguni Languages
Venue: arXiv cs.CL 2021
Year: 2021

Morphological Segmentation involves decomposing words into morphemes, the smallest meaning-bearing units of language. This is an important NLP task for morphologically-rich agglutinative languages such as the Southern African Nguni language group. In this paper, we investigate supervised and unsupervised models for two variants of morphological segmentation: canonical and surface segmentation. We train sequence-to-sequence models for canonical segmentation, where the underlying morphemes may not be equal to the surface form of the word, and Conditional Random Fields (CRF) for surface segmentation. Transformers outperform LSTMs with attention on canonical segmentation, obtaining an average F1 score of 72.5% across 4 languages. Feature-based CRFs outperform bidirectional LSTM-CRFs to obtain an average of 97.1% F1 on surface segmentation. In the unsupervised setting, an entropy-based approach using a character-level LSTM language model fails to outperforms a Morfessor baseline, while on some of the languages neither approach performs much better than a random baseline. We hope that the high performance of the supervised segmentation models will help to facilitate the development of better NLP tools for Nguni languages.
Modeling in Systems Engineering: Conceptual Time Representation
Venue: arXiv cs.SE 2021
Year: 2021

The increasing importance of such fields as embedded systems, pervasive computing, and hybrid systems control is increasing attention to the time-dependent aspects of system modeling. In this paper, we focus on modeling conceptual time. Conceptual time is time represented in conceptual modeling, where the notion of time does not always play a major role. Time modeling in computing is far from exhibiting a unified and comprehensive framework, and is often handled in an ad hoc manner. This paper contributes to the establishment of a broader understanding of time in conceptual modeling based on a software and system engineering model denoted thinging machine (TM). TM modeling is founded on a one-category ontology called a thimac (thing/machine) that is used to elaborate the design and analysis of ontological presumptions. The issue under study is a sample of abstract modeling domains as exemplified by time. The goal is to provide better understanding of the TM model by supplementing it with a conceptualization of time aspects. The results reveal new characteristics of time and related notions such as space, events, and system behavior.
A Combined Deep Learning based End-to-End Video Coding Architecture for YUV Color Space
Venue: arXiv cs.CV 2021
Year: 2021

Most of the existing deep learning based end-to-end video coding (DLEC) architectures are designed specifically for RGB color format, yet the video coding standards, including H.264/AVC, H.265/HEVC and H.266/VVC developed over past few decades, have been designed primarily for YUV 4:2:0 format, where the chrominance (U and V) components are subsampled to achieve superior compression performances considering the human visual system. While a broad number of papers on DLEC compare these two distinct coding schemes in RGB domain, it is ideal to have a common evaluation framework in YUV 4:2:0 domain for a more fair comparison. This paper introduces a new DLEC architecture for video coding to effectively support YUV 4:2:0 and compares its performance against the HEVC standard under a common evaluation framework. The experimental results on YUV 4:2:0 video sequences show that the proposed architecture can outperform HEVC in intra-frame coding, however inter-frame coding is not as efficient on contrary to the RGB coding results reported in recent papers.
Partition-Guided GANs
Venue: arXiv cs.LG 2021
Year: 2021

Despite the success of Generative Adversarial Networks (GANs), their training suffers from several well-known problems, including mode collapse and difficulties learning a disconnected set of manifolds. In this paper, we break down the challenging task of learning complex high dimensional distributions, supporting diverse data samples, to simpler sub-tasks. Our solution relies on designing a partitioner that breaks the space into smaller regions, each having a simpler distribution, and training a different generator for each partition. This is done in an unsupervised manner without requiring any labels. We formulate two desired criteria for the space partitioner that aid the training of our mixture of generators: 1) to produce connected partitions and 2) provide a proxy of distance between partitions and data samples, along with a direction for reducing that distance. These criteria are developed to avoid producing samples from places with non-existent data density, and also facilitate training by providing additional direction to the generators. We develop theoretical constraints for a space partitioner to satisfy the above criteria. Guided by our theoretical analysis, we design an effective neural architecture for the space partitioner that empirically assures these conditions. Experimental results on various standard benchmarks show that the proposed unsupervised model outperforms several recent methods.
Trust-based Blockchain Authorization for IoT
Venue: arXiv cs.CR 2021
Year: 2021

Authorization or access control limits the actions a user may perform on a computer system, based on predetermined access control policies, thus preventing access by illegitimate actors. Access control for the Internet of Things (IoT) should be tailored to take inherent IoT network scale and device resource constraints into consideration. However, common authorization systems in IoT employ conventional schemes, which suffer from overheads and centralization. Recent research trends suggest that blockchain has the potential to tackle the issues of access control in IoT. However, proposed solutions overlook the importance of building dynamic and flexible access control mechanisms. In this paper, we design a decentralized attribute-based access control mechanism with an auxiliary Trust and Reputation System (TRS) for IoT authorization. Our system progressively quantifies the trust and reputation scores of each node in the network and incorporates the scores into the access control mechanism to achieve dynamic and flexible access control. We design our system to run on a public blockchain, but we separate the storage of sensitive information, such as user's attributes, to private sidechains for privacy preservation. We implement our solution in a public Rinkeby Ethereum test-network interconnected with a lab-scale testbed. Our evaluations consider various performance metrics to highlight the applicability of our solution for IoT contexts.
SDAN: Squared Deformable Alignment Network for Learning Misaligned Optical Zoom
Venue: arXiv cs.CV 2021
Year: 2021

Deep Neural Network (DNN) based super-resolution algorithms have greatly improved the quality of the generated images. However, these algorithms often yield significant artifacts when dealing with real-world super-resolution problems due to the difficulty in learning misaligned optical zoom. In this paper, we introduce a Squared Deformable Alignment Network (SDAN) to address this issue. Our network learns squared per-point offsets for convolutional kernels, and then aligns features in corrected convolutional windows based on the offsets. So the misalignment will be minimized by the extracted aligned features. Different from the per-point offsets used in the vanilla Deformable Convolutional Network (DCN), our proposed squared offsets not only accelerate the offset learning but also improve the generation quality with fewer parameters. Besides, we further propose an efficient cross packing attention layer to boost the accuracy of the learned offsets. It leverages the packing and unpacking operations to enlarge the receptive field of the offset learning and to enhance the ability of extracting the spatial connection between the low-resolution images and the referenced images. Comprehensive experiments show the superiority of our method over other state-of-the-art methods in both computational efficiency and realistic details.
Deep ensembles based on Stochastic Activation Selection for Polyp Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Semantic segmentation has a wide array of applications ranging from medical-image analysis, scene understanding, autonomous driving and robotic navigation. This work deals with medical image segmentation and in particular with accurate polyp detection and segmentation during colonoscopy examinations. Several convolutional neural network architectures have been proposed to effectively deal with this task and with the problem of segmenting objects at different scale input. The basic architecture in image segmentation consists of an encoder and a decoder: the first uses convolutional filters to extract features from the image, the second is responsible for generating the final output. In this work, we compare some variant of the DeepLab architecture obtained by varying the decoder backbone. We compare several decoder architectures, including ResNet, Xception, EfficentNet, MobileNet and we perturb their layers by substituting ReLU activation layers with other functions. The resulting methods are used to create deep ensembles which are shown to be very effective. Our experimental evaluations show that our best ensemble produces good segmentation results by achieving high evaluation scores with a dice coefficient of 0.884, and a mean Intersection over Union (mIoU) of 0.818 for the Kvasir-SEG dataset. To improve reproducibility and research efficiency the MATLAB source code used for this research is available at GitHub: https://github.com/LorisNanni.
An NCAP-like Safety Indicator for Self-Driving Cars
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes a mechanism to assess the safety of autonomous cars. It assesses the car's safety in scenarios where the car must avoid collision with an adversary. Core to this mechanism is a safety measure, called Safe-Kamikaze Distance (SKD), which computes the average similarity between sets of safe adversary's trajectories and kamikaze trajectories close to the safe trajectories. The kamikaze trajectories are generated based on planning under uncertainty techniques, namely the Partially Observable Markov Decision Processes, to account for the partially observed car policy from the point of view of the adversary. We found that SKD is inversely proportional to the upper bound on the probability that a small deformation changes a collision-free trajectory of the adversary into a colliding one. We perform systematic tests on a scenario where the adversary is a pedestrian crossing a single-lane road in front of the car being assessed --which is, one of the scenarios in the Euro-NCAP's Vulnerable Road User (VRU) tests on Autonomous Emergency Braking. Simulation results on assessing cars with basic controllers and a test on a Machine-Learning controller using a high-fidelity simulator indicates promising results for SKD to measure the safety of autonomous cars. Moreover, the time taken for each simulation test is under 11 seconds, enabling a sufficient statistics to compute SKD from simulation to be generated on a quad-core desktop in less than 25 minutes.
GAVIN: Gaze-Assisted Voice-Based Implicit Note-taking
Venue: arXiv cs.HC 2021
Year: 2021

Annotation is an effective reading strategy people often undertake while interacting with digital text. It involves highlighting pieces of text and making notes about them. Annotating while reading in a desktop environment is considered trivial but, in a mobile setting where people read while hand-holding devices, the task of highlighting and typing notes on a mobile display is challenging. In this paper, we introduce GAVIN, a gaze-assisted voice note-taking application, which enables readers to seamlessly take voice notes on digital documents by implicitly anchoring them to text passages. We first conducted a contextual enquiry focusing on participants' note-taking practices on digital documents. Using these findings, we propose a method which leverages eye-tracking and machine learning techniques to annotate voice notes with reference text passages. To evaluate our approach, we recruited 32 participants performing voice note-taking. Following, we trained a classifier on the data collected to predict text passage where participants made voice notes. Lastly, we employed the classifier to built GAVIN and conducted a user study to demonstrate the feasibility of the system. This research demonstrates the feasibility of using gaze as a resource for implicit anchoring of voice notes, enabling the design of systems that allow users to record voice notes with minimal effort and high accuracy.
An early warning AI-powered portable system to reduce workload and inspect environmental damage after natural disasters
Venue: arXiv cs.RO 2021
Year: 2021

1.3 million household fires, 3,400 civilian deaths, and 23 billion dollars in damage, a fire department is called to respond every 24 seconds. Many firefighters are injured during search and rescue operations due to hidden dangers. Additionally, fire-retardant water runoff pollution can threaten human health. My goal is to develop a system to monitor calamity-induced environment damage to provide early-intelligence to incident-commanders. I have developed a multi-spectral sensing system to inspect air and water quality for safer and accessible hazardous environment operations. Key components include a) drone mounted with four sensors (gas sensors, thermal camera, GPS sensor, visual camera) and wireless communicator for inspection, b) AI-powered computer vision base-station to identify targets, c) low-cost, portable, spectral water quality analyzer and d) robotic retriever. The prototype demonstrates the potential for safer and more accessible search and rescue operations for fire-fighters and scientists. The gas sensor could identify thick smoke situations (thresholds > 400). The visual and thermal cameras detected hidden hot objects and sent images to AI-powered analyzer to identify and localize target with rescue GPS coordinates for robotic retrieval. Water quality was analyzed with spectral signatures to indicate turbidity levels that correlate with potential pollutants (threshold > 1.3). Prototype results were shown to the Sunnyvale fire department and received encouraging feedback. Future goals include monitoring firefighter health and overexertion with smart clothes.
Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey
Venue: arXiv cs.LG 2021
Year: 2021

Most of state of the art methods applied on time series consist of deep learning methods that are too complex to be interpreted. This lack of interpretability is a major drawback, as several applications in the real world are critical tasks, such as the medical field or the autonomous driving field. The explainability of models applied on time series has not gather much attention compared to the computer vision or the natural language processing fields. In this paper, we present an overview of existing explainable AI (XAI) methods applied on time series and illustrate the type of explanations they produce. We also provide a reflection on the impact of these explanation methods to provide confidence and trust in the AI systems.
An Evaluation of the Archive of Formal Proofs
Venue: arXiv cs.DL 2021
Year: 2021

The Archive of Formal Proofs (AFP) is an online repository of formal proofs for the Isabelle proof assistant. It serves as a central location for publishing, discovering, and viewing libraries of proofs. We conducted an online survey in November 2020 to assess the suitability of the website. In this report, we present and discuss the results, which showed that long-term users of the website are generally satisfied with the AFP but that there are a number of areas, such as navigation, search and script browsing, that need improvement.