Reviewerly/exHarmony huggingface dataset:
Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

Processing visual stimuli in a scene is essential for the human brain to make situation-aware decisions. These stimuli, which are prevalent subjects of diagnostic eye tracking studies, are commonly encoded as rectangular areas of interest (AOIs) per frame. Because it is a tedious manual annotation task, the automatic detection and annotation of visual attention to AOIs can accelerate and objectify eye tracking research, in particular for mobile eye tracking with egocentric video feeds. In this work, we implement two methods to automatically detect visual attention to AOIs using pre-trained deep learning models for image classification and object detection. Furthermore, we develop an evaluation framework based on the VISUS dataset and well-known performance metrics from the field of activity recognition. We systematically evaluate our methods within this framework, discuss potentials and limitations, and propose ways to improve the performance of future automatic visual attention detection methods.
PDANet: Self-Supervised Monocular Depth Estimation Using Perceptual and Data Augmentation Consistency

In recent studies, self-supervised learning methods have been explored for monocular depth estimation. They minimize the reconstruction loss of images instead of depth information as a supervised signal. However, existing methods usually assume that the corresponding points in different views should have the same color, which leads to unreliable unsupervised signals and ultimately damages the reconstruction loss during the training. Meanwhile, in the low texture region, it is unable to predict the disparity value of pixels correctly because of the small number of extracted features. To solve the above issues, we propose a network—PDANet—that integrates perceptual consistency and data augmentation consistency, which are more reliable unsupervised signals, into a regular unsupervised depth estimation model. Specifically, we apply a reliable data augmentation mechanism to minimize the loss of the disparity map generated by the original image and the augmented image, respectively, which will enhance the robustness of the image in the prediction of color fluctuation. At the same time, we aggregate the features of different layers extracted by a pre-trained VGG16 network to explore the higher-level perceptual differences between the input image and the generated one. Ablation studies demonstrate the effectiveness of each components, and PDANet shows high-quality depth estimation results on the KITTI benchmark, which optimizes the state-of-the-art method from 0.114 to 0.084, measured by absolute relative error for depth estimation.
MHNF: Multi-Hop Heterogeneous Neighborhood Information Fusion Graph Representation Learning

The attention mechanism enables graph neural networks (GNNs) to learn the attention weights between the target node and its one-hop neighbors, thereby improving the performance further. However, most existing GNNs are oriented toward homogeneous graphs, and in which each layer can only aggregate the information of one-hop neighbors. Stacking multilayer networks introduces considerable noise and easily leads to over smoothing. We propose here a multihop heterogeneous neighborhood information fusion graph representation learning method (MHNF). Specifically, we propose a hybrid metapath autonomous extraction model to efficiently extract multihop hybrid neighbors. Then, we formulate a hop-level heterogeneous information aggregation model, which selectively aggregates different-hop neighborhood information within the same hybrid metapath. Finally, a hierarchical semantic attention fusion model (HSAF) is constructed, which can efficiently integrate different-hop and different-path neighborhood information. In this fashion, this paper solves the problem of aggregating multihop neighborhood information and learning hybrid metapaths for target tasks. This mitigates the limitation of manually specifying metapaths. In addition, HSAF can extract the internal node information of the metapaths and better integrate the semantic information present at different levels. Experimental results on real datasets show that MHNF achieves the best or competitive performance against state-of-the-art baselines with only a fraction of 1/10 <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 1/100 parameters and computational budgets. Our code is publicly available at <uri>https://github.com/PHD-lanyu/MHNF</uri>
Comparative Performance Analysis of Optical Flow Algorithms for Anomaly Detection

Presently there is major interest in visual surveillance systems for crowd anomaly detection. Horn-Schunck, Lucas–Kanade and Farneback optical flow methods are used for the estimation of motion in the scene. The motion vectors; magnitude and orientation are analyzed in this work to detect anomaly in crowd. It helps classify the crowd as normal or abnormal, walking or running, vehicle entering in crowd. Various databases are evaluated to check the validity of the feature extraction process. In order to cluster the behaviour as normal or abnormal, Artificial Neural Network is used as a classifier. The results from Farneback optical flow estimation algorithm are promising for crowd behaviour understanding and anomaly detection.
Optimizing Rankings for Recommendation in Matching Markets

Based on the success of recommender systems in e-commerce and entertainment, there is growing interest in their use in matching markets like job search. While this holds potential for improving market fluidity and fairness, we show in this paper that naively applying existing recommender systems to matching markets is sub-optimal. Considering the standard process where candidates apply and then get evaluated by employers, we present a new recommendation framework to model this interaction mechanism and propose efficient algorithms for computing personalized rankings in this setting. We show that the optimal rankings need to not only account for the potentially divergent preferences of candidates and employers, but they also need to account for capacity constraints. This makes conventional ranking systems that merely rank by some local score (e.g., one-sided or reciprocal relevance) highly sub-optimal — not only for an individual user, but also for societal goals (e.g., low unemployment). To address this shortcoming, we propose the first method for jointly optimizing the rankings for all candidates in the market to explicitly maximize social welfare. In addition to the theoretical derivation, we evaluate the method both on simulated environments and on data from a real-world networking-recommendation system that we built and fielded at a large computer science conference.
Learning from EPI-Volume-Stack for Light Field image angular super-resolution

Light Field (LF) image angular super-resolution aims to synthesize a high angular resolution LF image from a low angular resolution one, and is drawing increased attention because of its wide applications. In order to reconstruct a high angular resolution LF image, many learning based LF image angular super-resolution methods have been proposed. However, most existing methods are based on LF Epipolar Plane Image or Epipolar Plane Image volume representation, which underuse the LF image structure. The LF view spatial correlation and neighboring LF views angular correlations which can reflect LF image structure are not fully explored, which reduces LF angular super-resolution quality. In order to alleviate this problem, this paper introduces an Epipolar Plane Image Volume Stack (EPI-VS) representation for LF angular super-resolution. The EPI-VS is constituted by arranging all LF views in a raster order, which benefits in exploring LF view spatial correlation and neighboring LF views angular correlations. Based on such representation, we further propose an LF angular super-resolution network. 3D convolutions are applied in the whole super-resolution network to better accommodate the input EPI-VS data and allow information propagation between two spatial and one directional dimensions of EPI-VS data. Extensive experiments on synthetic and real-world LF scenes demonstrate the effectiveness of the proposed network. Moreover, we also illustrate the superiority of our network by applying it in scene depth estimation task. • A light field image angular super-resolution network based on light field EPI-Volume-Stack representation is proposed. • A new light field EPI-Volume-Stack representation for light field image angular super-resolution is introduced to explore light field view spatial and angular correlations. • A learning based light field method is proposed, in which 3D convolutional operations are adopted to better accommodate EPIVolume-Stack data and allow information propagation between two spatial and one directional dimensions of EPI-Volume-Stack data. • Extensive experiments on real world scenes, synthetic scenes, and LF application validate the effectiveness of the proposed method in achieving a better super-resolution quality and restoring more texture details of synthesized light field views.
A Labeled GM-PHD Filter for Explicitly Tracking Multiple Targets

In this study, an explicit track continuity algorithm is proposed for multitarget tracking (MTT) based on the Gaussian mixture (GM) implementation of the probability hypothesis density (PHD) filter. Trajectory maintenance and multitarget state extraction in the GM-PHD filter have not been effectively integrated to date. To address this problem, we propose an improved GM-PHD filter. In this approach, the Gaussian components are classified and labeled, and multitarget state extraction is converted into multiple single-state extractions. This provides the identity label of the individual target and can shield against the negative effects of clutter in the prior density region on the estimates, thus realizing the integration of trajectory maintenance with state extraction in the GM-PHD filter. As no additional associated procedures are required, the overall real-time performance of the proposed filter is similar to or slightly lower than that of the basic GM-PHD filter. The results of numerical experiments demonstrate that the proposed approach can achieve explicit track continuity.
TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge

Real-time video analytics on the edge is challenging as the computationally constrained resources typically cannot analyse video streams at full fidelity and frame rate, which results in loss of accuracy. This paper proposes a Transprecise Object Detector (TOD) which maximises the real-time object detection accuracy on an edge device by selecting an appropriate Deep Neural Network (DNN) on the fly with negligible computational overhead. TOD makes two key contributions over the state of the art: (1) TOD leverages characteristics of the video stream such as object size and speed of movement to identify networks with high prediction accuracy for the current frames; (2) it selects the best-performing network based on projected accuracy and computational demand using an effective and low-overhead decision mechanism. Experimental evaluation on a Jetson Nano demonstrates that TOD improves the average object detection precision by 34.7% over the YOLOv4-tiny-288 model on average over the MOT17Det dataset. In the MOT17-05 test dataset, TOD utilises only 45.1% of GPU resource and 62.7% of the GPU board power without losing accuracy, compared to YOLOv4-416 model. We expect that TOD will maximise the application of edge devices to real-time object detection, since TOD maximises real-time object detection accuracy given edge devices according to dynamic input features without increasing inference latency in practice.
FAWOS: Fairness-Aware Oversampling Algorithm Based on Distributions of Sensitive Attributes

With the increased use of machine learning algorithms to make decisions which impact people's lives, it is of extreme importance to ensure that predictions do not prejudice subgroups of the population with respect to sensitive attributes such as race or gender.Discrimination occurs when the probability of a positive outcome changes across privileged and unprivileged groups defined by the sensitive attributes.It has been shown that this bias can be originated from imbalanced data contexts where one of the classes contains a much smaller number of instances than the other classes.It is also important to identify the nature of the imbalanced data, including the characteristics of the minority classes' distribution.This paper presents FAWOS: a Fairness-Aware oversampling algorithm which aims to attenuate unfair treatment by handling sensitive attributes' imbalance.We categorize different types of datapoints according to their local neighbourhood with respect to the sensitive attributes, identifying which are more difficult to learn by the classifiers.In order to balance the dataset, FAWOS oversamples the training data by creating new synthetic datapoints using the different types of datapoints identified.We test the impact of FAWOS on different learning classifiers and analyze which can better handle sensitive attribute imbalance.Empirically, we observe that this algorithm can effectively increase the fairness results of the classifiers while not neglecting the classification performance.
On-the-Fly Parallel Processing IP-Core for Image Blur Detection, Compression, and Chaotic Encryption Based on FPGA

This paper presents a 3 in 1 standalone FPGA system which can perform color image blur detection in parallel with compression and encryption. Both blur detection and compression are based on the 3-level Haar wavelet transform, which is used as a common building block to save the resources. The compression is based on performing the hard thresholding scheme followed by the Run Length Encoding (RLE) technique. The encryption is based on the 128-bit Advanced Encryption Standard (AES), which is considered one of the most secure algorithms. Moreover, the modified Lorenz chaotic system is combined with the AES to perform the Cipher Block Chaining (CBC) mode. The proposed system is realized using HDL and implemented using Xilinx on XC5VLX50T FPGA. The system has utilized only 25% of the available slices. Furthermore, the system can achieve a throughput of 3.458 Gbps, which is suitable for real-time applications. To validate the compression performance, the system has been tested with all the standard 256×256 images. It is shown that depending on the amount of details in the image, the system can achieve 30dB PSNR at compression ratios in the range of (0.08-0.38). The proposed system can be integrated with digital cameras to process the captured images on-the-fly prior to transmission or storage. Based on the application, the blurred images can be either marked for future enhancement or simply filtered out.
Leveraging Supervised Machine Learning Techniques for Identification of Malaria Cells using Blood Smears

Plasmodium parasite is identified as amenable for spreading a disease named Malaria. Under the stodgy method, the blood splotch is first smeared on the slide, scrutinized under the microscope, and parasites (which can cause malaria) in blood cells are detected. For beneficial parasite detection, image processing proves to be very much dominant. The reason for this is accuracy in the results. This research presents Malaria detection in blood smear images using supervised learning methods. This proposed method starts with the preprocessing in which images are resized and converted into grayscale. The thresholding technique is implemented to identify blobs for segmentation. For feature extraction, GoogLeNet is maneuvered, and the results of the classification show that this method has an accuracy of 95.8% for detecting malaria in blood smear images.
Automatic Checkerboard Detection for Robust Camera Calibration

Accurate checkerboard recognition is vital for basic camera calibration in many machine vision tasks. However, existing detectors usually encounter two problems. First, X-corner location precision is easily affected by undesired factors such as noise and distortion. Then, the pattern recovery requires manual input of the dimensions of the real corner matrix, and most recovery schemes can’t deal well with the pattern with occlusion or missing corners. In this paper we propose a novel CNN-based checkerboard detection framework to address these problems. This framework consists of three sub-modules: 1) an X-corner detection network to identify as many real corners as possible. 2) a sub-pixel refinement technique obtained from the geometric analysis of the image response map and grayscale to find precise corner locations. 3) a pattern recovery scheme to find the regular checkerboard layout even with high distortion and partial occlusion. Quantitative experimental results show that the proposed approach has higher accuracy and stronger robustness than state-of-the-art methods to both synthetic images and real-world camera calibration scenarios.
Predictability Classes for Forecasting Clients Behavior by Transactional Data

Nowadays, the task of forecasting the client’s behavior using his/her digital footprints is highly demanded. There are many approaches to predict the client’s next purchase or the next location visited that focus on achieving the best possible prediction quality in terms of different quality metrics. Within such approaches, the quality is however usually evaluated on the entire set of clients, without dividing them into classes with a different predictability rate of client’s behavior. In contrast to the approaches of this type, we propose a method for the identification of the client’s behaviour predictability class by means of a foreign trip in the next month by using only client’s historical transactional data. In a sense, this allows us to estimate the quality of forecasting the client’s foreign trip before the actual prediction procedure. Our experiments show that the approach is rather efficient and that the predictability classes obtained quite agree with the prediction quality classes found within the actual forecasting.
Multitask prediction of organ dysfunction in the intensive care unit using sequential subnetwork routing

Multitask learning (MTL) using electronic health records allows concurrent prediction of multiple endpoints. MTL has shown promise in improving model performance and training efficiency; however, it often suffers from negative transfer - impaired learning if tasks are not appropriately selected. We introduce a sequential subnetwork routing (SeqSNR) architecture that uses soft parameter sharing to find related tasks and encourage cross-learning between them.Using the MIMIC-III (Medical Information Mart for Intensive Care-III) dataset, we train deep neural network models to predict the onset of 6 endpoints including specific organ dysfunctions and general clinical outcomes: acute kidney injury, continuous renal replacement therapy, mechanical ventilation, vasoactive medications, mortality, and length of stay. We compare single-task (ST) models with naive multitask and SeqSNR in terms of discriminative performance and label efficiency.SeqSNR showed a modest yet statistically significant performance boost across 4 of 6 tasks compared with ST and naive multitasking. When the size of the training dataset was reduced for a given task (label efficiency), SeqSNR outperformed ST for all cases showing an average area under the precision-recall curve boost of 2.1%, 2.9%, and 2.1% for tasks using 1%, 5%, and 10% of labels, respectively.The SeqSNR architecture shows superior label efficiency compared with ST and naive multitasking, suggesting utility in scenarios in which endpoint labels are difficult to ascertain.
MULTI-AUV DYNAMIC MANEUVER DECISION-MAKING BASED ON INTUITIONISTIC FUZZY COUNTER-GAME AND FRACTIONAL-ORDER PARTICLE SWARM OPTIMIZATION

In this paper, a multi-AUV dynamic maneuver decision-making algorithm is studied based on intuitionistic fuzzy game and fractional-order Particle Swarm Optimization (PSO). Because of the weak communication condition and complex marine environment, a maneuver decision-making algorithm is usually hard to realize in real-time multi-AUV couter-game process. First, the weak communication condition is analyzed according to sonar and other equipment characteristics. Then, the multi-AUV maneuver attributes evaluation and maneuver decision-making modeling are investigated under the obtained weak communication constraints. Subsequently, a fractional-order PSO optimization method is proposed to solve the strategy optimization problem of multi-AUV maneuver decision-making process. At last, an example is presented to verify the effectiveness and superiority of the obtained algorithm.
A super resolution frontal face generation model based on 3DDFA and CBAM

• FFSR_GAN was proposed. • CBAM is introduced for the frontal face generation module. • Using FRB-GAN to achieve super-resolution face reconstruction. Pose and low resolution seriously affect the synthesis of high-quality frontal face images. With the development of deep learning, a large number of models based on the deep neural network are used to solve the problem of face pose and image super-resolution. However, the synthesis of the high-resolution frontal face is still a problem that has not been fully studied. Therefore, in this paper, we propose a method to realize image super-resolution and face frontal generation simultaneously. Specifically, we propose a frontal face model FFSR_GAN used to generate super-resolution. This model mainly solves the problem of low resolution and large face pose. There are two main improvements: 1) Aiming at the problem of artifacts in the image generated by the face frontal generation module, the face frontal generation module is designed based on 3DDFA and CBAM; 2) Aiming at the problem of low resolution in frontal face generation, a face super-resolution module is carefully designed, which is used for super-resolution of the generated frontal face. The method proposed in this paper solves the problem of face pose and super-resolution for the first time and improves the recognition accuracy of low-resolution and face images with larger posture. The experimental results on the existing public dataset prove the advantages of the FFSR_GAN model.
Viewing anti-immigrant hate online: An application of routine activity and Social Structure-Social Learning Theory

The increasingly prominent role of the Internet in the lives of Americans has resulted in more people coming into contact with various types of online content, including online hate material. One of the most common forms of online hate targets immigrants, seeking to position immigrants as threats to personal, national, economic, and cultural security. Given the recent rise in online hate targeting immigrants, this study examines factors that bring individuals into virtual contact with such material. Utilizing recently collected online survey data of American youth and young adults, we draw on insights from Routine Activity Theory and Social Structure-Social Learning Theory to understand exposure to anti-immigrant online hate material. Specifically, we consider how online routines, location in social structure, and social identity are associated with exposure. Results indicate that engaging in behaviors that can increase proximity to motivated offenders increases the likelihood of being exposed to anti-immigrant hate, as does engaging in online behaviors that bolster one's target suitability. Additionally, individuals who view Americanism as fundamental to their social identity are more apt to encounter anti-immigrant hate material on the Internet, as are those who are more dissatisfied with the current direction of the country.