Reviewerly/exHarmony huggingface dataset:
My View is the Best View: Procedure Learning from Egocentric Videos

Procedure learning involves identifying the key-steps and determining their logical order to perform a task. Existing approaches commonly use third-person videos for learning the procedure, making the manipulated object small in appearance and often occluded by the actor, leading to significant errors. In contrast, we observe that videos obtained from first-person (egocentric) wearable cameras provide an unobstructed and clear view of the action. However, procedure learning from egocentric videos is challenging because (a) the camera view undergoes extreme changes due to the wearer's head motion, and (b) the presence of unrelated frames due to the unconstrained nature of the videos. Due to this, current state-of-the-art methods' assumptions that the actions occur at approximately the same time and are of the same duration, do not hold. Instead, we propose to use the signal provided by the temporal correspondences between key-steps across videos. To this end, we present a novel self-supervised Correspond and Cut (CnC ) framework for procedure learning. CnC identifies and utilizes the temporal correspondences between the key-steps across multiple videos to learn the procedure. Our experiments show that CnC outperforms the state-of-the-art on the benchmark ProceL and CrossTask datasets by $$5.2\%$$ and $$6.3\%$$ , respectively. Furthermore, for procedure learning using egocentric videos, we propose the EgoProceL dataset consisting of 62 hours of videos captured by 130 subjects performing 16 tasks. The source code and the dataset are available on the project page https://sid2697.github.io/egoprocel/ .
Federated Learning-Empowered Disease Diagnosis Mechanism in the Internet of Medical Things: From the Privacy-Preservation Perspective

The deep integration of the Internet of Things (IoT) and the medical industry has given birth to the Internet of Medical Things (IoMT). In IoMT, physicians treat a patient's disease by analyzing patient data collected through mobile devices with the assistance of an artificial intelligence (AI)-empowered systems. However, the traditional AI technologies may lead to the leakage of patient privacy data due to its own design flaws. As a privacy-preserving federated learning (FL) can generate a global disease diagnosis model through multiparty collaboration. However, FL is still unable to resist inference attacks. In this article, to address such problems, we propose a privacy-enhanced disease diagnosis mechanism using FL for IoMT. Specifically, we first reconstruct medical data through a variational autoencoder and add differential privacy noise to it to resist inference attacks. These data are then used to train local disease diagnosis models, thereby preserving patients' privacy. Furthermore, to encourage participation in FL, we propose an incentive mechanism to provide corresponding rewards to participants. Experiments are conducted on the arrhythmia database Massachusetts Institute of Technology and Beth Israel Hospital (MIT-BIH). The experimental results show that the proposed mechanism reduces the probability of reconstructing patient medical data while ensuring high-precision heart disease diagnosis.
An Efficient and Robust Committee Structure for Sharding Blockchain

Nowadays, sharding is deemed a promising way to save traditional blockchain protocols from their low scalability. However, such a technique also brings several potential risks and a huge communication burden. An improper design may give rise to an inconsistent state among different committees. Further, the communication burden arising from cross-shard transactions, unfortunately, reduces the system's performance. In this paper, we first summarize five essential issues that all sharding blockchain designers face. For each issue, we discuss its key challenge and propose our suggested solutions. In order to break the performance bottlenecks, we design a committee structure and propose a reputation mechanism for selecting leaders. The term reputation in our design reflects each node's honest computation resources. In addition, we present a recovery procedure in case the leader is malicious. Theoretically, we prove that the system is robust under our design. Further simulation results also support this. In addition, the results show that selecting leaders by reputation can dramatically improve the system's performance.
DST: Dynamic Substitute Training for Data-free Black-box Attack

With the wide applications of deep neural network models in various computer vision tasks, more and more works study the model vulnerability to adversarial examples. For data-free black box attack scenario, existing methods are inspired by the knowledge distillation, and thus usually train a substitute model to learn knowledge from the target model using generated data as input. However, the substitute model always has a static network structure, which limits the attack ability for various target models and tasks. In this paper, we propose a novel dynamic substitute training attack method to encourage substitute model to learn better and faster from the target model. Specifically, a dynamic substitute structure learning strategy is proposed to adaptively generate optimal substitute model structure via a dy-namic gate according to different target models and tasks. Moreover, we introduce a task-driven graph-based structure information learning constrain to improve the quality of generated training data, and facilitate the substitute model learning structural relationships from the target model multiple outputs. Extensive experiments have been conducted to verify the efficacy of the proposed attack method, which can achieve better performance compared with the state-of-the-art competitors on several datasets. Project page: https://wxwangiris.github.io/DST
Neural Network-Based OFDM Receiver for Resource Constrained IoT Devices

Orthogonal Frequency Division Multiplexing (OFDM)-based waveforms are used for communication links in many current and emerging Internet of Things (IoT) applications, including the latest WiFi standards. For such OFDM-based transceivers, many core physical layer functions related to channel estimation, demapping, and decoding are implemented for specific choices of channel types and modulation schemes, among others. To decouple hard-wired choices from the receiver chain and thereby enhance the flexibility of IoT deployment in many novel scenarios without changing the underlying hardware, we explore a novel, modular Machine Learning (ML)-based receiver chain design. Here, ML blocks replace the individual processing blocks of an OFDM receiver, and we specifically describe this swapping for the legacy channel estimation, symbol demapping, and decoding blocks with Neural Networks (NNs). A unique aspect of this modular design is providing flexible allocation of processing functions to the legacy or ML blocks, allowing them to interchangeably coexist. Furthermore, we study the implementation cost-benefits of the proposed NNs in resource-constrained IoT devices through pruning and quantization, as well as emulation of these compressed NNs within Field Programmable Gate Arrays (FPGAs). Our evaluations demonstrate that the proposed modular NN-based receiver improves bit error rate of the traditional non-ML receiver by averagely 61 percent and 10 percent for the simulated and over-the-air datasets, respectively. We further show complexity-performance tradeoffs by presenting computational complexity comparisons between the traditional algorithms and the proposed compressed NNs.
2D GANs Meet Unsupervised Single-View 3D Reconstruction

Recent research has shown that controllable image generation based on pre-trained GANs can benefit a wide range of computer vision tasks. However, less attention has been devoted to 3D vision tasks. In light of this, we propose a novel image-conditioned neural implicit field, which can leverage 2D supervisions from GAN-generated multi-view images and perform the single-view reconstruction of generic objects. Firstly, a novel offline StyleGAN-based generator is presented to generate plausible pseudo images with full control over the viewpoint. Then, we propose to utilize a neural implicit function, along with a differentiable renderer to learn 3D geometry from pseudo images with object masks and rough pose initializations. To further detect the unreliable supervisions, we introduce a novel uncertainty module to predict uncertainty maps, which remedy the negative effect of uncertain regions in pseudo images, leading to a better reconstruction performance. The effectiveness of our approach is demonstrated through superior single-view 3D reconstruction results of generic objects. Code is available at http://cvlab.cse.msu.edu/project-gansvr.html .
An Optimal Fusion Method of Multiple Inertial Measurement Units Based on Measurement Noise Variance Estimation

At present, most inertial systems generally only contain a single inertial measurement unit (IMU). Considering the low cost and low accuracy of the micro-electromechanical system (MEMS)-IMU, it has attracted much attention to fuse multiple IMUs to improve the accuracy and robustness of the system. In this article, two online noise variance estimators based on second-order-mutual-difference (SOMD) algorithm are proposed for two redundant measurements and multiple redundant measurements, respectively. In addition, the unbiasedness and consistency of the estimators are proved. Using the proposed noise variance estimators, measurement noise variances of each sensor can be estimated in real time when multiple IMUs exist. Based on the estimated noise variance of each sensor, the weighted least squares (WLS) estimation method is used to generate the optimal virtual IMU (VIMU) in the observation domain. Finally, comparative simulations and the real-world experiment were conducted to evaluate the proposed online noise estimation algorithm. The simulation results demonstrate its superiority compared with other noise variance estimation methods, and the real-world experiment results show the effectiveness of the IMU fusion method based on the proposed noise variance estimation algorithm.
An Impact of YOLOv5 on Text Detection and Recognition System using TesseractOCR in Images/Video Frames

Text detection and recognition in images and videos are significant research areas in computer vision. A computer vision technology is used for smart city real-time traffic monitoring, and a security camera can simultaneously record the license plate information of suspected vehicles. The challenging task here is detecting the text images that are arbitrary oriented, such as aerial photographs and scene texts. Most complementary text detection and recognition methods are designed to identify text in images that are clear in the background and near-horizontal text. However, those methods will not be effective in detecting text in complex images and video streams. To address this issue, we propose a system that detects the text images using the YOLOv5s model, which effectively trains small-scale images and YOLOv5x for largescale images. TesseractOCR recognizes the detected text by converting the image to a string and storing it in CSV format. The experiment was carried out for ICDAR2013, ICDAR2015, and YVT images/frames. The results indicate that the proposed method using YOLOv5x effectively detects images/video frames with reasonably good accuracy, and the recognition rate is suitable for a near-horizontal image using TesseractOCR.
A Differential Measure of the Strength of Causation

We present the problem of measuring the strength of a causal interaction, starting from the linear perspective and generalizing to a nonlinear measure of causal influence. The proposed measure of causal strength is interpretable and we demonstrate that it may be estimated efficiently using Gaussian process regression. We validate our results on several examples and connect our results to the existing causal inference literature.
Compressing Models with Few Samples: Mimicking then Replacing

Few-sample compression aims to compress a big redundant model into a small compact one with only few samples. If we fine-tune models with these limited few samples directly, models will be vulnerable to overfit and learn almost nothing. Hence, previous methods optimize the compressed model layer-by-layer and try to make every layer have the same outputs as the corresponding layer in the teacher model, which is cumbersome. In this paper, we propose a new framework named Mimicking then Replacing (MiR) for few-sample compression, which firstly urges the pruned model to output the same features as the teacher's in the penultimate layer, and then replaces teacher's layers before penultimate with a well-tuned compact one. Unlike previous layer-wise reconstruction methods, our MiR optimizes the entire network holistically, which is not only simple and effective, but also unsupervised and general. MiR outperforms previous methods with large margins. Codes is available at https://github.com/cjnjuwhy/MiR.
FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs

Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than $$15\%$$ FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR .
Hand Gesture Recognition for Filipino Sign Language Under Different Backgrounds

The article implements a hand gesture Filipino Sign Language recognition model using Raspberry Pi. Numerous studies on Filipino Sign Language (FSL) frequently identify a letter with a glove and using a plain background, which may be challenging if implemented in a more complex background. Limited research on the implementation of YOLO-Lite and MobileNetV2 on FSL were also observed. Implementing YOLO-Lite for hand detection and MobileNetV2 for classification, the average accuracy achieved for differentiating 26 hand gestures, representing FSL letters, was 93.29%. The model demonstrated dependability in a variety of complex backgrounds. However, challenges in recognizing letters Q, J, and Z were encountered. Additionally, in letters N and M, due to their similar hand structures, N is sometimes mistakenly interpreted as M. The model developed by the researchers performed well and demonstrated better accuracy compared to a different model. The system was able to achieve higher accuracy while running on limited resources and in various environments.
Heterogeneous Trust-based Social Recommendation via Reliable and Informative Motif-based Attention

Recently, social recommender systems are promising to enhance the quality of recommendations by integrating user-user social networks and user-item bipartite networks. However, there are very little success in that direction. Pessimistic findings are ascribed mainly to three factors. (1) Very few works focus on the identification of implicit neighbors to overcome the sparsity problem of explicit links. Furthermore, these works do not con-sider higher-order and complex patterns of interactivity among users. (2) Very less number of trust-based social recommender systems integrate heterogeneous trust relationships, and this het-erogeneity is considered for explicit social links only. Moreover, these works ignore user-user heterogeneous trust relationships of higher-order network structure and user-item heterogeneous interactivity. (3) Existing works overlook the reliability (or lack of that) problem of links in higher-order and complex patterns of interactivity. To address the above mentioned challenges, we develop, a Graph <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\bar{C}$</tex> onvolutional <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\bar{N}$</tex> etworks via <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\bar{R}$</tex> eliable and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\bar{I}$</tex> nformative <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\bar{M}$</tex> otif-based Attention Model (CNRIM). To the best of our knowledge, it is the first work that investigates user-user heterogeneous trust relationships and user-item heterogeneous interactivity via reliable, informative motif-based attention mech-anisms. Varying reliability and informative motifs introduce the heterogeneity. The experiments on publicly available real-world datasets, and empirical analyses present the superiority of our model over popular baselines.
Hybrid Feature Selection Based on Principal Component Analysis and Grey Wolf Optimizer Algorithm for Arabic News Article Classification

The rapid growth of electronic documents has resulted from the expansion and development of internet technologies.Text-documents classification is a key task in natural language processing that converts unstructured data into structured form and then extract knowledge from it.This conversion generates a high dimensional data that needs further analusis using data mining techniques like feature extraction, feature selection, and classification to derive meaningful insights from the data.Feature selection is a technique used for reducing dimensionality in order to prune the feature space and, as a result, lowering the computational cost and enhancing classification accuracy.This work presents a hybrid filter-wrapper method based on Principal Component Analysis (PCA) as a filter approach to select an appropriate and informative subset of features and Grey Wolf Optimizer (GWO) as wrapper approach (PCA-GWO) to select further informative features.Logistic Regression (LR) is used as an elevator to test the classification accuracy of candidate feature subsets produced by GWO.Three Arabic datasets, namely Alkhaleej, Akhbarona, and Arabiya, are used to assess the efficiency of the proposed method.The experimental results confirm that the proposed method based on PCA-GWO outperforms the baseline classifiers with/without feature selection and other feature selection approaches in terms of classification accuracy.
Internet of Things Centric-Based Multiactivity Recognition in Smart Home Environment

In recent times, numerous human activity recognition (HAR) schemes have been proposed with embedding sensors, wearable devices, smart phones, and vision and ambient sensors. Though the systems have shown better performance they are mostly standalone and still lack the ability to share, host, and perform real-time analysis and visualization of activity data. The Internet of Things (IoT) paradigm has a solution to render the limitations and this will pave the way for HAR in the smart home environment. Thus in this article, an IoT-centric multiactivity recognition system is proposed and deployed on the cloud platform for activity data tracking in the smart home environment. The proposed system collects the real-time data collected using IMU sensors and transmitted to the IoT-Edge Server via Wi-Fi where the data has been fused and classified using light-weight deep learning models. This system has a provision of a Web-based dashboard which is helpful for the home dwellers to monitor the activities in the remote. The performance evaluation justified that the developed system can measure IoT-based activity recognition with greater efficiency in terms of accuracy and F1-score in a shorter response time as of deployment in the cloud platform to detect the activity.
Minimum Variance Embedded Random Vector Functional Link Network with Privileged Information

A teacher in a school plays significant role in classroom while teaching the students. Similarly, learning via privileged information (LUPI) gives extra information generated by a teacher to ‘teach’ the learning algorithm while training. This paper proposes minimum variance embedded random vector functional link network with privileged information (MVRVFL+). The proposed MVRVFL+ minimizes the intraclass variance of the training data and uses privileged information paradigm which provides the additional knowledge during the training of the model. The proposed MVRVFL+ classification model is evaluated on 43 benchmark UCI datasets. From the experimental analysis, the proposed MVRVFL+ showed best average accuracy and emerged as the lowest average rank classifier among the baseline models.
Efficient Fall Detection for a Healthcare Robot System Based on 3-Axis Accelerometer and Depth Sensor Fusion with LSTM Networks

Fall detection is one of the most important functions for a healthcare robot system because falls are very dangerous for older people and might lead to death if failed to provide prompt and adequate treatment. In this paper, we propose an efficient fall detection method based on 3-axis accelerometer and depth sensor fusion. LSTM networks are applied to handle temporal information. Simple low-level motion and pose features are obtained from each sensor data, and then fed into the LSTM networks that can learn high-level feature representations to classify falls from other daily life activities. Also, various learning tricks are combined to improve the performance. Experimental results show that the proposed fall detection method outperforms existing methods.
Revealing the Distributional Vulnerability of Discriminators by Implicit Generators

In deep neural learning, a discriminator trained on in-distribution (ID) samples may make high-confidence predictions on out-of-distribution (OOD) samples. This triggers a significant matter for robust, trustworthy and safe deep learning. The issue is primarily caused by the limited ID samples observable in training the discriminator when OOD samples are unavailable. We propose a general approach for fine-tuning discriminators by implicit generators (FIG). FIG is grounded on information theory and applicable to standard discriminators without retraining. It improves the ability of a standard discriminator in distinguishing ID and OOD samples by generating and penalizing its specific OOD samples. According to the Shannon entropy, an energy-based implicit generator is inferred from a discriminator without extra training costs. Then, a Langevin dynamic sampler draws specific OOD samples for the implicit generator. Lastly, we design a regularizer fitting the design principle of the implicit generator to induce high entropy on those generated OOD samples. The experiments on different networks and datasets demonstrate that FIG achieves the state-of-the-art OOD detection performance.