Reviewerly/exHarmony huggingface dataset:
Learning multi-view visual correspondences with self-supervision

Stereo-based 3D reconstruction requires to match features across images captured from slightly different viewing angles to recover 3D coordinates of the image pixels. Despite the workload of collecting data, annotating matched pixels requires also heavy labor. As recent researches for self-supervised representation learning has gained great progress, learning multi-view visual correspondences from large scale raw videos serves as an alternative. However, existing methods which benefit from contrastive learning tend to neglect false negative samples when matching between adjacent frames in a video, leading to sub-optimal optimization for visual features. In this paper, we propose a contrastive learning framework that construct self-supervision by semi-global visual correspondence to alleviate learning degradation when false negatives are involved in training. Our learning framework consists of pixel-level contrastive learning via patch reconstruction and patch-level contrastive learning cross videos. We also introduce saliency guidance to extract salient regions from video frames to further reduce potential false negatives. By optimizing the model with the proposed semi-global contrastive learning method, learned representations are forced to be discriminative and robust. Experiments demonstrate that our proposed method outperforms previous self-supervised methods on video object segmentation tasks. Moreover, when compared to fully-supervised algorithms designed for specific tasks, our proposed method also achieves competitive results.
Eager Falsification for Accelerating Robustness Verification of Deep Neural Networks

Formal robustness verification of deep neural networks (DNNs) is a promising approach for achieving a provable reliability guarantee to AI-enabled software systems. Limited scalability is one of the main obstacles to the verification problem. In this paper, we propose eager falsification to accelerate the robustness verification of DNNs. It divides the verification problem into a set of independent subproblems and solves them in descending order of their falsification probabilities. Once a subproblem is falsified, the verification terminates with a conclusion that the network is not robust. We introduce a notion of label affinity to measure the falsification probability and present an approach to computing the probability based on symbolic interval propagation. Our approach is orthogonal to existing verification techniques. We integrate it into four state-of-the-art verification tools, i.e., MIPVerify, Neurify, DeepZ, and DeepPoly, and conduct extensive experiments on 8 benchmark datasets. The experimental results show that our approach can significantly improve these tools by up to 200x speedup when the perturbation distance is in a reasonable range.
Dictionary learning for multivariate geochemical anomaly detection for mineral exploration targeting

Dictionary learning is usually used to train an overcomplete dictionary composed of basis vectors (i.e., atoms) to best sparsely represent the training dataset. The sparse representation of a data point is a sparse linear combination of the atoms derived from the well-trained overcomplete dictionary. The sparse representations of all the training data points constitute a supporting set of the population distribution of the training dataset. In this paper, geochemical exploration data was used as the training data of the five dictionary learning algorithms to train overcomplete dictionaries. The sparse representations of all the geochemical data points were used to describe the (complex) geochemical background, and the Euclidean distance between a data point and its sparse representation was used to express the anomality of the data point in multivariate geochemical anomaly detection. For demonstration purposes, the Chengde district in Hebei Province (China) was used as the case study area, and five dictionary learning models were established for multivariate geochemical anomaly detection for gold mineral exploration targeting. The performances of the five dictionary learning models were compared with that of the k-nearest neighbor (KNN) model, combined KNN model, and Gaussian mixture model (GMM) in gold mineral exploration targeting. The results show that (a) the five dictionary learning models are superior to the KNN model, the combined KNN model, and the GMM model either in terms of receiver operating characteristic curve (ROC) or in terms of area under the curve (AUC), and (b) the gold prospective areas optimally delineated in the study area accounts for a small proportion of the area, but contains most of the known gold deposits, and have spatial correlation with the favorable ore-forming factors such as the outer contact zone of multistage magmatic intrusion and the NE-trending fault zone. Therefore, the dictionary learning algorithms are feasible techniques of multivariate geochemical anomaly detection for mineral exploration targeting. It is worth to further test the usefulness of the dictionary learning algorithms in different areas with complex geochemical backgrounds.
Aggregate interactive learning for RGB-D salient object detection

The task of salient object detection is to find the most noticeable areas in the image. On the one hand, most of the existing RGB-D saliency object detection requires additional networks to process depth features, and the sub-networks that process depth features rely too much on the RGB network, resulting in higher computational costs. On the other hand, when dealing with multi-scale features, most models tend to produce information loss, the semantic information representation ability is weak, and good detection results cannot be achieved, which limits its practical application. Firstly, this paper proposes a strategy of aggregation and interaction to extract edge features, depth features and salient features, while maintaining local details, fully extracting global information. Secondly, in the learning process of high-level features, depth features and salient features are extracted at the same time, which reduces the complexity of the network and does not require the additional sub-networks. On the other hand, the deformable convolution network is used to solve the multi-scale problem to ensure that more detailed feature information is extracted. Finally, taking advantage of the complementarity between features, using the one-to-one feature fusion module, the problem of information redundancy in the feature fusion process is solved, and the fused features can accurately locate the salient results with clear details. The experimental results on six datasets show that compared with other state-of-the-art algorithms, the algorithm in this paper has excellent performance.
Multistrategy boosted multicolony whale virtual parallel optimization approaches

A multistrategy boosted multicolony whale optimization algorithm (MSMCWOA) is proposed. First, humpback whales are divided into different subcolonies based on various behavioral mechanisms of whale predation. Each subcolony evolves independently to ensure the diversity of the algorithm population. At the same time, local knowledge or experience generated from each subcolony is periodically provided to the global storage space. When the diversity of colonies decreases sharply, a hierarchical decision-making model based on information sharing is stimulated. Global storage space applies optimal information to guide subcolonies to jump out of potential local optimization. In addition, individuals in different subcolonies choose different strategies to communicate with each other with certain probability. The ability of knowledge acquisition and dissemination and the distance between individuals are considered to improve the effectiveness of information exchange and space search. Twenty-three unimodal and multimodal benchmark functions are tested and compared with state-of-the-art algorithms, and the experimental results show that the MSMCWOA has a better performance in terms of convergence rate and stability. The MSMCWOA is also successfully applied to the optimization design of the pressure vessel and tension/compression spring, and the result shows the superior performance of the MSMCWOA in solving project optimization problems.
Multiple Thermal Sensor Array Fusion Toward Enabling Privacy-Preserving Human Monitoring Applications

Human-centric applications of a single thermal sensor array (TSA) have performed extremely well in many areas. However, most of these works have not yet reached the real applicability stage of the Internet of Things (IoT) applications. The main limitation of deploying such systems on a large scale is the challenge of fusing multiple TSAs to cover a wide inspection area, e.g., smart homes, hospitals, and many other domestic environments. On the other hand, objects that appear in the low-resolution thermal images acquired from TSA have low intraclass variations and high interclass similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This article proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout to enable further human-centred IoT applications to run in the cloud. Besides, a privacy improvement on utilizing these sensors in IoT applications is proposed. The proposed approach is evaluated with comprehensive experiments on different sensor placements and domestic environment conditions. This article shows an average performance of 92.5% accuracy using various machine learning techniques and use case scenarios.
Federated learning-based intrusion detection in SDN-enabled IIoT networks

Witnessing the explosion in the number of Internet of Things (IoTs) in industries, Software Defined Networking (SDN) is considered as a flexible, efficient, and programmable approach for network management and security policy enforcement. Particularly, it is more suitable in the context of industrial Internet of Things (IIoT) network comprising heterogeneous devices. Meanwhile, the demand of ensuring cyber threat resistance has more become the serious concern from both academia and industry due to incidents, cyberattacks, personal data breaches reported recently. Many intrusion detection systems (IDS) leverage the advances in machine learning (ML) to build the more efficient attack detector against the unknown malicious actions in the network. Such an approach requires gathering a large amount of network traffic for model training in a centralized platform. It obviously violates the data privacy protection since the network traffic is sensitive information if accessed and used by a third party. To take advantage of private network data from various sources for mutually training detection model, federated learning (FL) is recently introduced as a solution that can address the problem of violating data privacy for ML-based cybersecurity solution during training phase. Thus, this work introduces the FL approach for IDS to facilitate the privacy preserving in model training while collaboratively maintaining the efficiency of attack detection in IIoT context with the leverage of SDN.
Hybrid Feature-Based Invasive Ductal Carcinoma Classification in Breast Histopathology Images

Breast cancer appears to be the most dangerous disease infecting women. Invasive ductal carcinoma is the most prevalent type of cancer that affects the breast. One in eight ladies suffer from carcinoma and even it leads to the removal of their breast. This study describes a hybrid feature-based approach for detecting ınvasive ductal carcinoma in WSI images. An optimal set of hybrid features is extracted from the images using deep learning models and handcrafted feature extraction techniques. In this work, we have done a comparison of feature sets considering the combination of handcrafted features like grey level co-occurrence matrix, Gabor filter and linear binary pattern with different deep learning models. A set consists of 9 handcrafted and 1000 deep learning features that are combined and used to classify the image patches as IDC or non-IDC by using a k-NN classifier. The experiment was performed on a Kaggle dataset, having breast histopathology images with 277,524 patches. Experimental results have shown that the handcrafted features with the MobileNet model features gave better classification accuracy, F1-score, precision and recall (91%, 87%, 88% and 85%, respectively).
Enumerating Fair Packages for Group Recommendations

Package-to-group recommender systems recommend a set of unified items to a group of people. Different from conventional settings, it is not easy to measure the utility of group recommendations because it involves more than one user. In particular, fairness is crucial in group recommendations. Even if some members in a group are substantially satisfied with a recommendation, it is undesirable if other members are ignored to increase the total utility. Many methods for evaluating and applying the fairness of group recommendations have been proposed in the literature. However, all these methods maximize the score and output only one package. This is in contrast to conventional recommender systems, which output several (e.g., top-K) candidates. This can be problematic because a group can be dissatisfied with the recommended package owing to some unobserved reasons, even if the score is high. To address this issue, we propose a method to enumerate fair packages efficiently. Our method furthermore supports filtering queries, such as top-K and intersection, to select favorite packages when the list is long. We confirm that our algorithm scales to large datasets and can balance several aspects of the utility of the packages.
MoESR: Blind Super-Resolution using Kernel-Aware Mixture of Experts

Modern deep learning super-resolution approaches have achieved remarkable performance where the low-resolution (LR) input is a degraded high-resolution (HR) image by a fixed known kernel i.e. kernel-specific super-resolution (SR). However, real images often vary in their degradation kernels, thus a single kernel-specific SR approach does not often produce accurate HR results. Recently, degradation-aware networks are introduced to generate blind SR results for unknown kernel conditions. They can restore images for multiple blur kernels. However, they have to compromise in quality compared to their kernel-specific counterparts. To address this issue, we propose a novel blind SR method called Mixture of Experts Super-Resolution (MoESR), which uses different experts for different degradation kernels. A broad space of degradation kernels is covered by kernel-specific SR networks (experts). We present an accurate kernel prediction method (gating mechanism) by evaluating the sharpness of images generated by experts. Based on the predicted kernel, our most suited expert network is selected for the input image. Finally, we fine-tune the selected network on the test image itself to leverage the advantage of internal learning. Our experimental results on standard synthetic datasets and real images demonstrate that MoESR outperforms state-of-the-art methods both quantitatively and qualitatively. Especially for the challenging ×4 SR task, our PSNR improvement of 0.93 dB on the DIV2KRK dataset is substantial <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Scalable Graph Topology Learning via Spectral Densification

Graph learning plays an important role in many data mining and machine learning tasks, such as manifold learning, data representation and analysis, dimensionality reduction, data clustering, and visualization, etc. In this work, we introduce a highly-scalable spectral graph densification approach (GRASPEL) for graph topology learning from data. By limiting the precision matrix to be a graph-Laplacian-like matrix, our approach aims to learn sparse undirected graphs from potentially high-dimensional input data. A very unique property of the graphs learned by GRASPEL is that the spectral embedding (or approximate effective-resistance) distances on the graph will encode the similarities between the original input data points. By leveraging high-performance spectral methods, sparse yet spectrally-robust graphs can be learned by identifying and including the most spectrally-critical edges into the graph. Compared with prior state-of-the-art graph learning approaches, GRASPEL is more scalable and allows substantially improving computing efficiency and solution quality of a variety of data mining and machine learning applications, such as manifold learning, spectral clustering (SC), and dimensionality reduction (DR).
Revocable identity-based fully homomorphic signature scheme with signing key exposure resistance

Fully homomorphic signature schemes in identity-based settings can provide authenticity, homomorphism, and non-repudiation as do traditional digital signatures, while simplifying the public key infrastructure (PKI) requirements, in which each user in the system can use his or her identity as a public key. As identity-based systems (IBS) have a natural link between unique identity information and the user, allowing user revocation is usually more difficult in IBS than in PKI settings. Although several revocable identity-based fully homomorphic signature (RIBFHS) schemes have been proposed, these schemes are vulnerable to signing key exposure. With this study, we are the first to consider a realistic threat signing key exposure in RIBFHS systems. In addition, we introduce a new security definition of RIBFHS with signing key exposure resistance. Then, we employ Agrawal's left–right lattices and delegation technology in fixed dimensions to construct a new RIBFHS scheme over the lattice with regularly broadcasted update keys, which not only resists signing key exposure, but also meets scalability and context hiding properties. Finally, we prove that our construction is existentially unforgeable against chosen message attacks under the standard short integer solution (SIS) assumption in the random oracle model.
Multi-branch Neural Networks for Video Anomaly Detection in Adverse Lighting and Weather Conditions

Automated anomaly detection in surveillance videos has attracted much interest as it provides a scalable alternative to manual monitoring.Most existing approaches achieve good performance on clean benchmark datasets recorded in well-controlled environments.However, detecting anomalies is much more challenging in the real world.Adverse weather conditions like rain or changing brightness levels cause a significant shift in the input data distribution, which in turn can lead to the detector model incorrectly reporting high anomaly scores.Additionally, surveillance cameras are usually deployed in evolving environments such as a city street of which the appearance changes over time because of seasonal changes or roadworks.The anomaly detection model will need to be updated periodically to deal with these issues.In this paper, we introduce a multi-branch model that is equipped with a trainable preprocessing step and multiple identical branches for detecting anomalies during day and night as well as in sunny and rainy conditions.We experimentally validate our approach on a distorted version of the Avenue dataset and provide qualitative results on real-world surveillance camera data.Experimental results show that our method outperforms the existing methods in terms of detection accuracy while being faster and more robust on scenes with varying visibility.
Parsing Line Chart Images Using Linear Programming

This paper proposes a method for automatically recovering data from chart images. In particular we focus on the task of estimating line charts, as the most common chart type, in a fully automatic way that handles line occlusions, as well as lines of different styles, e.g. dashed or dotted. For this, we first train a single semantic segmentation network to predict probability maps for each different line styles. We then construct a graph based on this output and formulate the line tracing task as a minimum-cost-flow problem, optimizing a cost function using linear programming. From the traced lines, the axes, and text labels, we recover the numerical values used to generate the chart. In experiments on six datasets, containing both synthesized and crawled images, we show significant improvements over prior work.
Content-Adaptive and Attention-Based Network for Hand Gesture Recognition

For hand gesture recognition, recurrent neural networks and 3D convolutional neural networks are the most commonly used methods for learning the spatial–temporal features of gestures. The calculation of the hidden state of the recurrent neural network at a specific time is determined by both input at the current time and the output of the hidden state at the previous time, therefore limiting its parallel computation. The large number of weight parameters that need to be optimized leads to high computational costs associated with 3D convolution-based methods. We introduced a transformer-based network for hand gesture recognition, which is a completely self-attentional architecture without any convolution or recurrent layers. The framework classifies hand gestures by focusing on the sequence information of the whole gesture video. In addition, we introduced an adaptive sampling strategy based on the video content to reduce the input of gesture-free frames to the model, thus reducing computational consumption. The proposed network achieved 83.2% and 93.8% recognition accuracy on two publicly available benchmark datasets, NVGesture and EgoGesture datasets, respectively. The results of extensive comparison experiments show that our proposed approach outperforms the existing state-of-the-art gesture recognition systems.
Head Motion Controlled Wheelchair for Physically Disabled People

Physically disabled people face difficulties in daily life because of their body impairment from their birth or due to an accident or illness. The project’s goal is to design a wheelchair that could function for a disable-person who cannot move other parts of the body correctly, keeping their words in mind with the help of head movements. Medical equipment manufactured to assist disabled peoples are very complicated, limited, and costly. A head motion controlled wheelchair is an intelligent wheelchair with facilities for navigating, recognizing obstacles, and moving automatically by managing detectors and motions. The prototype of the wheelchair performs head motion through a microcontroller. Furthermore, data processing is performed with the help of an accelerometer. The controller filters the indication and allows the action of the wheelchair for its navigation. The ultrasound detector helps to resist impediments. Usually, it is expensive, but we have designed it at an inadequate cost so that ordinary people from underdeveloped or developing countries can use it. The system memorizes the head gesture for further referencing it as the stable gesture or “neutral position” after identifying the start signal. The dc motors will drive the wheelchair during the gesture of control mode. The motors will not work, and consequently, the wheelchair will not run when the head is neutral.