Reviewerly/exHarmony huggingface dataset:
Multi-Instance Multi-Scale CNN for Medical Image Classification

Deep learning for medical image classification faces three major challenges: (1) the number of annotated medical images for training are usually small; (2) regions of interest (ROIs) are relatively small with unclear boundaries in the whole medical images, and may appear in arbitrary positions across the x, y (and also z in 3D images) dimensions. However often only labels of the whole images are annotated, and localized ROIs are unavailable; and (3) ROIs in medical images often appear in varying sizes (scales). We approach these three challenges with a Multi-Instance Multi-Scale (MIMS) CNN: (1) We propose a multi-scale convolutional layer, which extracts patterns of different receptive fields with a shared set of convolutional kernels, so that scale-invariant patterns are captured by this compact set of kernels. As this layer contains only a small number of parameters, training on small datasets becomes feasible; (2) We propose a "top-k pooling" to aggregate the feature maps in varying scales from multiple spatial dimensions, allowing the model to be trained using weak annotations within the multiple instance learning (MIL) framework. Our method is shown to perform well on three classification tasks involving two 3D and two 2D medical image datasets.
Improving Uncertainty Estimation in Convolutional Neural Networks Using Inter-rater Agreement

Modern neural networks are pushing the boundaries of medical image classification. For some tasks in dermatology, state of the art models are able to beat human experts in terms of accuracy and type I/II errors. However, in the domain of medical applications, models should also be evaluated on how well they capture uncertainty in samples and labels. This aspect is key to building trust in computer-assisted systems, otherwise largely considered to be black boxes by their users. A common practice in supervised learning is to collect multiple evaluations per sample, which is particularly useful when inter-rater agreement is expected to be low. At the same time, model training traditionally uses label fusion, such as majority voting, to produce a single label for each sample. In this paper, we experimentally show that models trained to predict skin conditions become overconfident when this approach is used; i.e. the probability estimates of the model exceeds the true correctness likelihood. Additionally, we show that a better calibrated model is obtained when training with a label sampling scheme that takes advantage of inter-rater variability during training. The calibration improvements come at no cost in terms of model accuracy. Our approach is combined and contrasted with other recent techniques in uncertainty estimation. All experiments are evaluated on a proprietary dataset consisting of 31017 images of skin, where up to 12 experts have diagnosed each image.
Blockchain-Based Approach for e-Health Data Access Management with Privacy Protection

Blockchain is a technology that offers the ability to create new business models and solves trust issues in a more efficient way. It can lead to many research opportunities and business innovations. Academia and industry proposed many blockchain based software solutions within a wide range of domains. In this paper we present a system design where blockchain technology is proposed to be used in the healthcare system, where the vital information regarding the medical analyses are shared between hospitals, medical clinics and research institutes based on access policies defined by the patients. In order to protect confidential data, our solution involves the use of two types of chains: a private one, the sidechain, which keeps information about real ID of the patients, and a public one, the mainchain, which stores information about patients' health data marked with a temporary ID. To test it, we developed the design using Hyperledger Fabric framework. Presented experimental results show good performance of the system in relation to the following metrics: 1) the time needed to identify the medical data for a particular patient, and 2) the mainchain propagation time of all the blocks within the peer to peer network.
Explore-Exploit Graph Traversal for Image Retrieval

We propose a novel graph-based approach for image retrieval. Given a nearest neighbor graph produced by the global descriptor model, we traverse it by alternating between exploit and explore steps. The exploit step maximally utilizes the immediate neighborhood of each vertex, while the explore step traverses vertices that are farther away in the descriptor space. By combining these two steps we can better capture the underlying image manifold, and successfully retrieve relevant images that are visually dissimilar to the query. Our traversal algorithm is conceptually simple, has few tunable parameters and can be implemented with basic data structures. This enables fast real-time inference for previously unseen queries with minimal memory overhead. Despite relative simplicity, we show highly competitive results on multiple public benchmarks, including the largest image retrieval dataset that is currently publicly available. Full code for this work is available here: https://github.com/layer6ai-labs/egt.
An Ultra-Low Energy Human Activity Recognition Accelerator for Wearable Health Applications

Human activity recognition (HAR) has recently received significant attention due to its wide range of applications in health and activity monitoring. The nature of these applications requires mobile or wearable devices with limited battery capacity. User surveys show that charging requirement is one of the leading reasons for abandoning these devices. Hence, practical solutions must offer ultra-low power capabilities that enable operation on harvested energy. To address this need, we present the first fully integrated custom hardware accelerator (HAR engine) that consumes 22.4 μJ per operation using a commercial 65 nm technology. We present a complete solution that integrates all steps of HAR , i.e., reading the raw sensor data, generating features, and activity classification using a deep neural network (DNN). It achieves 95% accuracy in recognizing 8 common human activities while providing three orders of magnitude higher energy efficiency compared to existing solutions.
Predicting Merge Conflicts in Collaborative Software Development

Background. During collaborative software development, developers often use branches to add features or fix bugs. When merging changes from two branches, conflicts may occur if the changes are inconsistent. Developers need to resolve these conflicts before completing the merge, which is an error-prone and time-consuming process. Early detection of merge conflicts, which warns developers about resolving conflicts before they become large and complicated, is among the ways of dealing with this problem. Existing techniques do this by continuously pulling and merging all combinations of branches in the background to notify developers as soon as a conflict occurs, which is a computationally expensive process. One potential way for reducing this cost is to use a machine-learning based conflict predictor that filters out the merge scenarios that are not likely to have conflicts, i.e.safe merge scenarios.Aims. In this paper, we assess if conflict prediction is feasible.Method. We design a classifier for predicting merge conflicts, based on 9 light-weight Git feature sets. To evaluate our predictor, we perform a large-scale study on 267,657 merge scenarios from 744 GitHub repositories in seven programming languages.Results. Our results show that we achieve high f1-scores, varying from 0.95 to 0.97 for different programming languages, when predicting safe merge scenarios. The f1-score is between 0.57 and 0.68 for the conflicting merge scenarios.Conclusions. Predicting merge conflicts is feasible in practice, especially in the context of predicting safe merge scenarios as a pre-filtering step for speculative merging.
Verification of high-dimensional entanglement generated in quantum interference

Entanglement and quantum interference are key ingredients in a variety of quantum information processing tasks. Harnessing the generation and characterization of entanglement in high-dimensional state spaces is a necessary prerequisite towards practical quantum protocols. Here, we use quantum interference on a beam splitter to engineer hyperentanglement in polarization and discrete frequency degrees of freedom (DOF). We show how independent measurements of polarization and frequency DOF allow for the verification of high-dimensional entanglement in the combined state space. These results may indicate new paths towards practical exploitation of entanglement stored in multiple degrees of freedom, in particular in the context of high-dimensional quantum information processing protocols.
Blockchain Technologies and Remittances: From Financial Inclusion to Correspondent Banking

Since their emergence, blockchain technologies have shown potential for financial inclusion and the formalization of remittances. Recently, regulators and practitioners have studied the capabilities of blockchain technologies to streamline and, potentially, replace the infrastructure underpinning cross-border payments and remittances, i.e., correspondent banking. Correspondent Banking Relationships, also called “Nostro-Vostro accounts,” are continuous bilateral arrangements that enable banks to provide services in countries where they do not directly operate. After the Global Financial Crisis, this infrastructure has undergone “de-risking,” i.e., a reduction of correspondent accounts and their concentration in fewer financial institutions, with especially detrimental effects on costs and speed of retail cross-border remittances. The existing literature has mostly focused on the point of sale of remittances, often overlooking correspondent banking. This paper, in contrast, connects remittances, blockchain technologies, and correspondent banking with the growing interest of critical social science in the significance of payment infrastructures for the constitution and configuration of money, finance, and markets. By unpacking the critical case of Ripple, this paper shows that blockchain applications to remittances focus on profits, risks, costs, interoperability, “trapped liquidity,” and “idle capital” in correspondent banking accounts, rather than on financial inclusion per se. In so doing, this paper contributes to critical social studies literature on the formalization of remittances, understood as the transformation of remittances into a market frontier. Blockchain applications are shown to foster, rather than resist, remittances formalization, and they are presently being incorporated into existing infrastructures, business models, and regulatory structures. Rather than representing radically alternative monetary systems, blockchain technologies are the latest iteration of technologies heralding frictionless capitalism. Lastly, this paper shows the tensions and ambiguities inherent to interoperability and formalization. Blockchain technologies are dynamic in a way that problematizes dichotomies such formal-informal and mainstream-alternative. Hence, rather than providing a quantitative assessment of the impact of blockchain technologies, this paper investigates the ambiguities and tensions in the political economy and imaginaries inscribed in the materiality and design of blockchain-enabled payment systems.
Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach

Human behavior expression and experience are inherently multimodal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the user’s states (e.g., engagement) are therefore needed. Most of the existing works that try to build classifiers for the user’s states assume that the data to train the models are fully labeled. Nevertheless, data labeling is costly and tedious, and also prone to subjective interpretations by the human coders. This is even more pronounced when the data are multi-modal (e.g., some users are more expressive with their facial expressions, some with their voice). Thus, building models that can accurately estimate the user’s states during an interaction is challenging. To tackle this, we propose a novel multi-modal active learning (AL) approach that uses the notion of deep reinforcement learning (RL) to find an optimal policy for active selection of the user’s data, needed to train the target (modality-specific) models. We investigate different strategies for multi-modal data fusion, and show that the proposed model-level fusion coupled with RL outperforms the feature-level and modality-specific models, and the naïve AL strategies such as random sampling, and the standard heuristics such as uncertainty sampling. We show the benefits of this approach on the task of engagement estimation from real-world child-robot interactions during an autism therapy. Importantly, we show that the proposed multi-modal AL approach can be used to efficiently personalize the engagement classifiers to the target user using a small amount of actively selected user’s data.
Underwater Color Restoration Using U-Net Denoising Autoencoder

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial sectors. However, the automatic extraction of information using software tools is hindered by the characteristics of water which degrade the quality of captured videos. As a contribution for restoring the color of underwater images, Underwater Denoising Autoencoder (UDAE) model is developed using a denoising autoencoder with U-Net architecture. The proposed network takes into consideration the accuracy and the computation cost to enable realtime implementation on underwater visual tasks using end-to-end autoencoder network. Underwater vehicles perception is improved by reconstructing captured frames; hence obtaining better performance in underwater tasks. Related learning methods use generative adversarial networks (GANs) to generate color corrected underwater images, and to our knowledge this paper is the first to deal with a single autoencoder capable of producing same or better results. Moreover, image pairs are constructed for training the proposed network, where it is hard to obtain such dataset from underwater scenery. At the end, the proposed model is compared to a state-of-the-art method.
Evaluating Internet of Medical Things (IoMT)-Based Systems from a Human-Centric Perspective

Internet of Things (IoT)-based systems are rapidly evolving, offering promising technological, financial and social prospects in multiple domains, including healthcare. Medical systems, integrated with IoT features, can support healthcare activities, such as the remote monitoring of patients, the real-time diagnosis of medical issues, etc., improving the quality of the provided services, and enhancing the associated human users’ satisfaction. These systems fall within the context of the Internet of Medical Things (IoMT). However, concerns, stemming from the human users, must be considered during the design of such systems, since they are transformed into criticalities, that restrict system functionality. In this work, stepping on a human-centric perspective of designing IoMT systems, we analyze the human concerns, associated to the systems’ usage, and the criticalities, that are extracted from these concerns. In addition, we identify specific criticality types, classifying the criticalities into them. Focus is given on the Remote Elderly Monitoring System (REMS) and the Smart Ambulance System (SAS) use cases, that belong to the healthcare domain, combining IoMT technologies with classic healthcare practices. Based on these systems, we explore and validate the applicability of the human-centric perspective, and examine whether concerns and criticality types are addressed. This results in the identification of ways to further improve such systems, and consequently, in greater satisfaction, and acceptance from the end-users.
Modular Lightweight Network for Road Object Detection Using a Feature Fusion Approach

This article presents a modular lightweight network model for road objects detection, such as car, pedestrian, and cyclist, especially when they are far away from the camera and their sizes are small. Great advances have been made for the deep networks, but small objects detection is still a challenging task. In order to solve this problem, a majority of existing methods utilize complicated network or bigger image size, which generally leads to higher computation cost. The proposed network model is referred to as modular feature fusion detector (MFFD), using a fast and efficient network architecture for detecting small objects. The contribution lies in the following aspects: 1) two base modules have been designed for efficient computation: a) Front module reduces the information loss from raw input images and b) Tinier module decreases the model size and computation cost, while ensuring the detection accuracy; 2) by stacking the base modules, we design a context features fusion framework for multiscale object detection; and 3) the proposed method is efficient in terms of model size and computation cost, which is applicable for resource-limited devices, such as embedded systems for advanced driver-assistance systems (ADASs). Comparisons with the state-of-the-art on the challenging KITTI dataset reveal the superiority of the proposed method. Especially, 100 ft/s can be achieved on the embedded GPUs such as Jetson TX2.
A Hybrid Model for Bitcoin Prices Prediction using Hidden Markov Models and Optimized LSTM Networks

With the recent advances in the Blockchain technology, and due to its decentralized nature, it has been a much considered approach for solving issues in the Internet of Things (IoT) sector, in particular, for IoT payment platforms. As Machine-to-Machine (M2M) payments are fundamental in the IoT economy, the development of Blockchain-based payment platforms, using cryptocurrency, is continuously increasing as it enables a pure M2M, secure and private financial transactions. Unlike traditional assets, cryptocurrencies have a higher index of volatility, which makes it essential to understand the movement of their prices, as a first step to optimize Blockchain-based M2M payment transactions. In this paper, we propose a novel hybrid model that deals with this challenge from a descriptive, as well as predictive points of view. We use Hidden Markov Models to describe cryptocurrencies historical movements to predict future movements with Long Short Term Memory networks. To evaluate the proposed hybrid model, we have chosen 2-minute frequency Bitcoin data from Coinbase exchange market. Our proposed model proved its effectiveness compared to traditional time-series forecasting models, ARIMA, as well as a conventional LSTM.
A Delay Metric for Video Object Detection: What Average Precision Fails to Tell

Average precision (AP) is a widely used metric to evaluate detection accuracy of image and video object detectors. In this paper, we analyze the object detection from video and point out that mAP alone is not sufficient to capture the temporal nature of video object detection. To tackle this problem, we propose a comprehensive metric, Average Delay (AD), to measure and compare detection delay. To facilitate delay evaluation, we carefully select a subset of ImageNet VID, which we name as ImageNet VIDT with an emphasis on complex trajectories. By extensively evaluating a wide range of detectors on VIDT, we show that most methods drastically increase the detection delay but still preserve mAP well. In other words, mAP is not sensitive enough to reflect the temporal characteristics of a video object detector. Our results suggest that video object detection methods should be evaluated with a delay metric, particularly for latency-critical applications such as autonomous vehicle perception.
Bidirectional One-Shot Unsupervised Domain Mapping

We study the problem of mapping between a domain A, in which there is a single training sample and a domain B, for which we have a richer training set. The method we present is able to perform this mapping in both directions. For example, we can transfer all MNIST images to the visual domain captured by a single SVHN image and transform the SVHN image to the domain of the MNIST images. Our method is based on employing one encoder and one decoder for each domain, without utilizing weight sharing. The autoencoder of the single sample domain is trained to match both this sample and the latent space of domain B. Our results demonstrate convincing mapping between domains, where either the source or the target domain are defined by a single sample, far surpassing existing solutions. Our code is made publicly available at https://github.com/tomercohen11/BiOST.
Revisiting Radial Distortion Absolute Pose

To model radial distortion there are two main approaches; either the image points are undistorted such that they correspond to pinhole projections, or the pinhole projections are distorted such that they align with the image measurements. Depending on the application, either of the two approaches can be more suitable. For example, distortion models are commonly used in Structure-from-Motion since they simplify measuring the reprojection error in images. Surprisingly, all previous minimal solvers for pose estimation with radial distortion use undistortion models. In this paper we aim to fill this gap in the literature by proposing the first minimal solvers which can jointly estimate distortion models together with camera pose. We present a general approach which can handle rational models of arbitrary degree for both distortion and undistortion.
Corner Detection Using Second-Order Generalized Gaussian Directional Derivative Representations

Corner detection is a critical component of many image analysis and image understanding tasks, such as object recognition and image matching. Our research indicates that existing corner detection algorithms cannot properly depict the difference between edges and corners and this results in wrong corner detections. In this paper, the capability of second-order generalized (isotropic and anisotropic) Gaussian directional derivative filters to suppress Gaussian noise is evaluated. The second-order generalized Gaussian directional derivative representations of step edge, L-type corner, Y- or T-type corner, X-type corner, and star-type corner are investigated and obtained. A number of properties for edges and corners are discovered which enable us to propose a new image corner detection method. Finally, the criteria on detection accuracy and average repeatability under affine image transformation, JPEG compression, and noise degradation, and the criteria on region repeatability are used to evaluate the proposed detector against nine state-of-the-art methods. The experimental results show that our proposed detector outperforms all the other tested detectors.
A Collaborative Recommendation System for Online Courses Recommendations

In this paper, we present a collaborative recommender system that recommends online courses for students based on similarities of students' course history. The proposed system employs data mining techniques to discover patterns between courses. Consequently, we have noticed that clustering students into similar groups based on their respective course selections play a vital role in generating association rules of high quality when compared with the association rules generated using the whole set of courses and students. In particular, the Apriori algorithm was used to generate association rules; once using the whole dataset and once using the clusters which are formed based on students' choices of courses. The results reveal that the coverage of the rules generated on clusters are better. Also, to assess the effect of course dependency on recommendations, we applied the SPADE algorithm on course sequences. The results are in harmony with the results obtained when Apriori was applied.