Reviewerly/exHarmony huggingface dataset:
DoubleX: Statically Detecting Vulnerable Data Flows in Browser Extensions at Scale

Browser extensions are popular to enhance users' browsing experience. By design, they have access to security- and privacy-critical APIs to perform tasks that web applications cannot traditionally do. Even though web pages and extensions are isolated, they can communicate through messages. Specifically, a vulnerable extension can receive messages from another extension or web page, under the control of an attacker. Thus, these communication channels are a way for a malicious actor to elevate their privileges to the capabilities of an extension, which can lead to, e.g., universal cross-site scripting or sensitive user data exfiltration. To automatically detect such security and privacy threats in benign-but-buggy extensions, we propose our static analyzer DoubleX. DoubleX defines an Extension Dependence Graph (EDG), which abstracts extension code with control and data flows, pointer analysis, and models the message interactions within and outside of an extension. This way, we can leverage this graph to track and detect suspicious data flows between external actors and sensitive APIs in browser extensions. We evaluated DoubleX on 154,484 Chrome extensions, where it flags 278 extensions as having a suspicious data flow. Overall, we could verify that 89% of these flows can be influenced by external actors (i.e., an attacker). Based on our threat model, we subsequently demonstrate exploitability for 184 extensions. Finally, we evaluated DoubleX on a labeled vulnerable extension set, where it accurately detects almost 93% of known flaws.
Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation

Volumetric deep learning approach towards stereo matching aggregates a cost volume computed from input left and right images using 3D convolutions. Recent works showed that utilization of extracted image features and a spatially varying cost volume aggregation complements 3D convolutions. However, existing methods with spatially varying operations are complex, cost considerable computation time, and cause memory consumption to increase. In this work, we construct Guided Cost volume Excitation (GCE) and show that simple channel excitation of cost volume guided by image can improve performance considerably. Moreover, we propose a novel method of using top-k selection prior to soft-argmin disparity regression for computing the final disparity estimate. Combining our novel contributions, we present an end-to-end network that we call Correlate-and-Excite (CoEx). Extensive experiments of our model on the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate the effectiveness and efficiency of our model and show that our model outperforms other speed-based algorithms while also being competitive to other state-of-the-art algorithms. Codes will be made available at https://github.com/antabangun/coex.
Deadline-aware Dynamic Resource Management in Serverless Computing Environments

Serverless computing enables rapid application development and deployment by composing loosely coupled microservices at a scale. This emerging paradigm greatly unburdens the users of cloud environments, from the need to provision and manage the underlying cloud resources. With this shift in responsibility, the cloud provider faces the challenge of providing acceptable performance to the user without compromising on reliability, while having minimal knowledge of the application requirements. Sub-optimal resource allocations, specifically the CPU resources, could result in the violation of performance requirements of applications. Further, the fine-grained serverless billing model only charges for resource usage in terms of function execution time. At the same time, the provider has to maintain the underlying infrastructure in always-on mode to facilitate asynchronous function calls. Thus, achieving optimum utilization of cloud resources without compromising on application requirements is of high importance to the provider. Most of the current works only focus on minimizing function execution times caused by delays in infrastructure set up and reducing resource costs for the end-user. However, in this paper, we focus on both the provider and user's perspective and propose a function placement policy and a dynamic resource management policy for applications deployed in serverless computing environments. The policies minimize the resource consumption cost for the service provider while meeting the user's application requirement, i.e., deadline. The proposed solutions are sensitive to deadline and efficiently increase the resource utilization for the provider, while dynamically managing resources to improve function response times. We implement and evaluate our approach through simulation using ContainerCloudSim toolkit. The proposed function placement policy when compared with baseline scheduling techniques can reduce resource consumption by up to three times. The dynamic resource allocation policy when evaluated with a fixed resource allocation policy and a proportional CPU-shares policy shows improvements of up to 25% in meeting the required function deadlines.
Mixed, Augmented and Virtual, Reality Applied to the Teaching of Mathematics for Architects

This paper examines the possibilities of Mixed Reality, the combination of two emerging technologies—Augmented Reality and Virtual Reality—in university education. For this purpose, an object was elaborated in Mixed Reality that underwent the evaluation of 44 first-year students from the degree in architecture who were enrolled in the subject “Mathematical Foundations for Architecture.” The instrument utilized was based on the TAM model, which analyzes the degree of acceptance of the technology used. The analysis of the responses provided by students supported the 23 hypotheses formulated in this study. It was found that MR significantly influences the perceived usefulness and ease of use. The results imply that MR utilization has positive effects on the mathematical teaching-learning processes in architecture from the students’ perception of their mastery of technology. It becomes necessary to offer support to those university teachers who promote the use of active MR-based methodologies in classrooms.
GOO: A Dataset for Gaze Object Prediction in Retail Environments

One of the most fundamental and information-laden actions humans do is to look at objects. However, a survey of current works reveals that existing gaze-related datasets annotate only the pixel being looked at, and not the boundaries of a specific object of interest. This lack of object an-notation presents an opportunity for further advancing gaze estimation research. To this end, we present a challenging new task called gaze object prediction, where the goal is to predict a bounding box for a person’s gazed-at object. To train and evaluate gaze networks on this task, we present the Gaze On Objects (GOO) dataset. GOO is composed of a large set of synthetic images (GOO-Synth) supplemented by a smaller subset of real images (GOO-Real) of people looking at objects in a retail environment. Our work establishes extensive baselines on GOO by re-implementing and evaluating selected state-of-the-art models on the task of gaze following and domain adaptation. Code is available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> on github.
Shape Sensing Using Two Outer Cores of Multicore Fiber and Optical Frequency Domain Reflectometer

A method, i.e., vector projections, using two outer cores of multicore fiber without calibration and OFDR was proposed to realize three-dimensional (3D) shape sensing. A complement for apparent curvature vector method using three outer cores of multicore fiber was also proposed to determine the viable fiber cores arrangement for shape sensing. Compared with apparent curvature vector method, the vector projections method based on two outer cores of multicore fiber decreased the number of fiber cores, which eliminating the requirement for fiber cores arrangement. Based on these methods, a reconstructed shape with a higher accuracy was obtained based on integrating all fiber shape from all cores combinations. The maximum root-mean-square errors (RMSEs) of end position for the vector projections and apparent curvature vector method were 13.1 and 12.4 mm, while a higher accuracy, i.e., maximum RMSEs of 11.9 and 12.1 mm, was obtained based on the integrated fiber shapes.
Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations

Abstract Deep learning has an increasing impact to assist research, allowing, for example, the discovery of novel materials. Until now, however, these artificial intelligence techniques have fallen short of discovering the full differential equation of an experimental physical system. Here we show that a dynamical neural network, trained on a minimal amount of data, can predict the behavior of spintronic devices with high accuracy and an extremely efficient simulation time, compared to the micromagnetic simulations that are usually employed to model them. For this purpose, we re-frame the formalism of Neural Ordinary Differential Equations to the constraints of spintronics: few measured outputs, multiple inputs and internal parameters. We demonstrate with Neural Ordinary Differential Equations an acceleration factor over 200 compared to micromagnetic simulations for a complex problem – the simulation of a reservoir computer made of magnetic skyrmions (20 minutes compared to three days). In a second realization, we show that we can predict the noisy response of experimental spintronic nano-oscillators to varying inputs after training Neural Ordinary Differential Equations on five milliseconds of their measured response to a different set of inputs. Neural Ordinary Differential Equations can therefore constitute a disruptive tool for developing spintronic applications in complement to micromagnetic simulations, which are time-consuming and cannot fit experiments when noise or imperfections are present. Our approach can also be generalized to other electronic devices involving dynamics.
Classification and Visualisation of Normal and Abnormal Radiographs; A Comparison between Eleven Convolutional Neural Network Architectures

This paper investigates the classification of radiographic images with eleven convolutional neural network (CNN) architectures (GoogleNet, VGG-19, AlexNet, SqueezeNet, ResNet-18, Inception-v3, ResNet-50, VGG-16, ResNet-101, DenseNet-201 and Inception-ResNet-v2). The CNNs were used to classify a series of wrist radiographs from the Stanford Musculoskeletal Radiographs (MURA) dataset into two classes—normal and abnormal. The architectures were compared for different hyper-parameters against accuracy and Cohen’s kappa coefficient. The best two results were then explored with data augmentation. Without the use of augmentation, the best results were provided by Inception-ResNet-v2 (Mean accuracy = 0.723, Mean kappa = 0.506). These were significantly improved with augmentation to Inception-ResNet-v2 (Mean accuracy = 0.857, Mean kappa = 0.703). Finally, Class Activation Mapping was applied to interpret activation of the network against the location of an anomaly in the radiographs.
Hand-eye calibration for 2D laser profile scanners using straight edges of common objects

This paper presents a method for calibrating a 2D profile laser scanner mounted on an industrial articulated robot; a task also known as the hand-eye calibration problem. The challenge of recovering the transformation matrix, from the robot's flange coordinate system to the scanner's coordinate system, lies in the lack of sufficient 3D information, as only 2D data is available. The task is typically performed using precision calibration specimens such as spheres, disks, and planes or using additional external devices such as cameras and 3D sensors. Here, we present an approach based on detecting straight edges found in common objects. Points extracted from the same edge, under various robot poses, are used to solve the calibration problem using a two-phase least-squares strategy, where rotation is recovered first, followed by translation. The process is semi-autonomous, requires minimal laborious and error-prone manual operations; its setup effort is small, because common objects can be used instead of costly precision gauges or external devices; it does not require large number of samples and it is simple to reason about, implement and compute.
GSDet: Object Detection in Aerial Images Based on Scale Reasoning

Variations in both object scale and style under different capture scenes ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , downtown, port) greatly enhance the difficulties associated with object detection in aerial images. Although ground sample distance (GSD) provides an apparent clue to address this issue, no existing object detection methods have considered utilizing this useful prior knowledge. In this paper, we propose the first object detection network to incorporate GSD into the object detection modeling process. More specifically, built on a two-stage detection framework, we adopt a GSD identification subnet converting the GSD regression into a probability estimation process, then combine the GSD information with the sizes of Regions of Interest (RoIs) to determine the physical size of objects. The estimated physical size can provide a powerful prior for detection by reweighting the weights from the classification layer of each category to produce RoI-wise enhanced features. Furthermore, to improve the discriminability among categories of similar size and make the inference process more adaptive, the scene information is also considered. The pipeline is flexible enough to be stacked on any two-stage modern detection framework. The improvement over the existing two-stage object detection methods on the DOTA dataset demonstrates the effectiveness of our method.
Smart Home Security Solutions using Facial Authentication and Speaker Recognition through Artificial Neural Networks

In this paper, a holistic solution for Smart Home Security is implemented which helps in improving privacy and security using two independent and emerging technologies of facial authentication and speech recognition. With the help of the proposed application, the user will be able to monitor his home through his mobile phone/tablet/PC. This method involves facial recognition by taking a real-time feed of the person at the door and then analysis of the live feed is carried out where the face recognised is authenticated with the data of owners in the database which matches the face to a name. Speech recognition has been used to doubly check the output of facial authentication. The entire process is carried out with the help of neural networks. If there's an unauthorized person at the door, an alert will be triggered and the owner will get a notification of this unauthorized access and would get to choose whether they want to add the person to their database or not. The overall accuracy of the proposed model is 82.71% with an accuracy of 87.5 % for Facial Authentication and 84.62 % for Speaker Authentication. Along with this, the main novelty for the research is to identify faces through masks which will help to properly verify the identity of the person and would prove to be beneficial not only in the current COVID-19 scenario but also in cases of thefts and burglaries by alerting the owner about the anomaly. Thus this smart security system can be extended to applications like banks, malls, offices, etc., and shall not be limited to only homes.
Common Investigation Process Model for Internet of Things Forensics

Internet of Things Forensics (IoTFs) is a new discipline in digital forensics science used in the detection, acquisition, preservation, rebuilding, analyzing, and the presentation of evidence from IoT environments. IoTFs discipline still suffers from several issues and challenges that have in the recent past been documented. For example, heterogeneity of IoT infrastructures has mainly been a key challenge. The heterogeneity of the IoT infrastructures makes the IoTFs very complex, and ambiguous among various forensic domain. This paper aims to propose a common investigation processes for IoTFs using the metamodeling method called Common Investigation Process Model (CIPM) for IoTFs. The proposed CIPM consists of four common investigation processes: i) preparation process, ii) collection process, iii) analysis process and iv) final report process. The proposed CIPM can assist IoTFs users to facilitate, manage, and organize the investigation tasks.
Resource-Aware Session Types for Digital Contracts

Programming digital contracts comes with unique challenges, which include (i) expressing and enforcing protocols of interaction, (ii) controlling resource usage, and (iii) preventing the duplication or deletion of a contract's assets. This article presents the design and type-theoretic foundation of Nomos, a programming language for digital contracts that addresses these challenges. To express and enforce protocols, Nomos is based on shared binary session types. To control resource usage, Nomos employs automatic amortized resource analysis. To prevent the duplication or deletion of assets, Nomos uses a linear type system. A monad integrates the effectful session-typed language with a general-purpose functional language. Nomos' prototype implementation features linear-time type checking and efficient type reconstruction that includes automatic inference of resource bounds via off-the-shelf linear optimization. The effectiveness of the language is evaluated with case studies on implementing common smart contracts such as auctions, elections, and currencies. Nomos is completely formalized, including the type system, a cost semantics, and a transactional semantics to deploy Nomos contracts on a blockchain. The type soundness proof ensures that protocols are followed at run-time and that types establish sound upper bounds on the resource consumption, ruling out re-entrancy and out-of-gas vulnerabilities.
Self-Supervised Visual Representations Learning by Contrastive Mask Prediction

Advanced self-supervised visual representation learning methods rely on the instance discrimination (ID) pretext task. We point out that the ID task has an implicit semantic consistency (SC) assumption, which may not hold in unconstrained datasets. In this paper, we propose a novel contrastive mask prediction (CMP) task for visual representation learning and design a mask contrast (MaskCo) framework to implement the idea. MaskCo contrasts region-level features instead of view-level features, which makes it possible to identify the positive sample without any assumptions. To solve the domain gap between masked and unmasked features, we design a dedicated mask prediction head in MaskCo. This module is shown to be the key to the success of the CMP. We evaluated MaskCo on training datasets beyond ImageNet and compare its performance with MoCo V2 [4]. Results show that MaskCo achieves comparable performance with MoCo V2 using ImageNet training dataset, but demonstrates a stronger performance across a range of downstream tasks when COCO or Conceptual Captions are used for training. MaskCo provides a promising alternative to the ID-based methods for self-supervised learning in the wild.
Personalized Federated Learning with Clustering: Non-IID Heart Rate Variability Data Application

While machine learning techniques are being applied to various fields for their exceptional ability to find complex relations in large datasets, the strengthening of regulations on data ownership and privacy is causing increasing difficulty in its application to medical data. In light of this, Federated Learning has recently been proposed as a solution to train on private data without breach of confidentiality. This conservation of privacy is particularly appealing in the field of healthcare, where patient data is highly confidential. However, many studies have shown that its assumption of Independent and Identically Distributed data is unrealistic for medical data. In this paper, we propose Personalized Federated Cluster Models, a hierarchical clustering-based FL process, to predict Major Depressive Disorder severity from Heart Rate Variability. By allowing clients to receive more personalized model, we address problems caused by non-IID data, showing an accuracy increase in severity prediction. This increase in performance may be sufficient to use Personalized Federated Cluster Models in many existing Federated Learning scenarios.
Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment

An important scenario for image quality assessment (IQA) is to evaluate image restoration (IR) algorithms. The state-of-the-art approaches adopt a full-reference paradigm that compares restored images with their corresponding pristine-quality images. However, pristine-quality images are usually unavailable in blind image restoration tasks and real-world scenarios. In this paper, we propose a practical solution named degraded-reference IQA (DR-IQA), which exploits the inputs of IR models, degraded images, as references. Specifically, we extract reference information from degraded images by distilling knowledge from pristine-quality images. The distillation is achieved through learning a reference space, where various degraded images are encouraged to share the same feature statistics with pristine-quality images. And the reference space is optimized to capture deep image priors that are useful for quality assessment. Note that pristine-quality images are only used during training. Our work provides a powerful and differentiable metric for blind IRs, especially for GAN-based methods. Extensive experiments show that our results can even be close to the performance of full-reference settings.
Development of three-dimensional pavement texture measurement technique using surface structured light projection

To realize active, non-contact, high-precision and full-field measurement of pavement texture, an innovative surface structured light projection (SSLP) based technique is developed in this paper. Firstly, the sinusoidal intensity encoded surface structured light is projected through the optical fiber interference, and the high-resolution industrial camera is used to capture the deformed fringe pattern modulated by the measured pavement. Then, a pre-processing method is proposed to suppress the noise and uneven background illumination in the captured fringe image. After that, a two-dimensional continuous wavelet transform method (2D-CWTM) is executed, and the wrapped phase can be demodulated accurately from the wavelet ridge. Finally, the 2π steps in the wrapped phase are removed by a phase unwrapper, and the measured pavement texture elevation can be obtained according to the phase-height mapping relationship. Both laboratory and field tests demonstrate the functionality and effectiveness of the proposed SSLP technique in pavement texture measurement.
Adversarial Attacks on Deep Learning Based mmWave Beam Prediction in 5G And Beyond

Deep learning provides powerful means to learn from spectrum data and solve complex tasks in 5G and beyond such as beam selection for initial access (IA) in mmWave communications. To establish the IA between the base station (e.g., gNodeB) and user equipment (UE) for directional transmissions, a deep neural network (DNN) can predict the beam that is best slanted to each UE by using the received signal strengths (RSSs) from a subset of possible narrow beams. While improving the latency and reliability of beam selection compared to the conventional IA that sweeps all beams, the DNN itself is susceptible to adversarial attacks. We present an adversarial attack by generating adversarial perturbations to manipulate the over-the-air captured RSSs as the input to the DNN. This attack reduces the IA performance significantly and fools the DNN into choosing the beams with small RSSs compared to jamming attacks with Gaussian or uniform noise.