Reviewerly/exHarmony huggingface dataset:
Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization

We target at the task of weakly-supervised action localization (WSAL), where only video-level action labels are available during model training. Despite the recent progress, existing methods mainly embrace a localization-by-classification paradigm and overlook the fruitful fine-grained temporal distinctions between video sequences, thus suffering from severe ambiguity in classification learning and classification-to-localization adaption. This paper argues that learning by contextually comparing sequence-to-sequence distinctions offers an essential inductive bias in WSAL and helps identify coherent action instances. Specifically, under a differentiable dynamic programming formulation, two complementary contrastive objectives are designed, including Fine-grained Sequence Distance (FSD) contrasting and Longest Common Subsequence (LCS) contrasting, where the first one considers the relations of various action/background proposals by using match, insert, and delete operators and the second one mines the longest common subsequences between two videos. Both contrasting modules can enhance each other and jointly enjoy the merits of discriminative action-background separation and alleviated task gap between classification and localization. Extensive experiments show that our method achieves state-of-the-art performance on two popular benchmarks. Our code is available at https://github.com/MengyuanChen21/CVPR2022-FTCL.
Revisiting the “Video” in Video-Language Understanding

What makes a video task uniquely suited for videos, beyond what can be understood from a single image? Building on recent progress in self-supervised image-language models, we revisit this question in the context of video and language tasks. We propose the atemporal probe (ATP), a new model for video-language analysis which provides a stronger bound on the baseline accuracy of multimodal models constrained by image-level understanding. By applying this model to standard discriminative video and language tasks, such as video question answering and text-to-video retrieval, we characterize the limitations and potential of current video-language benchmarks. We find that understanding of event temporality is often not necessary to achieve strong or state-of-the-art performance, even compared with recent large-scale video-language models and in contexts intended to benchmark deeper video-level understanding. We also demonstrate how ATP can improve both video-language dataset and model design. We describe a technique for leveraging ATP to better disentangle dataset subsets with a higher concentration of temporally challenging data, improving benchmarking efficacy for causal and temporal understanding. Further, we show that effectively integrating ATP into full video-level temporal models can improve efficiency and state-of-the-art accuracy. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Project website: https://stanfordvl.github.io/atp-revisit-video-lang/
Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches

Deep learning has substantially boosted the performance of Monocular Depth Estimation (MDE), a critical component in fully vision-based autonomous driving (AD) systems (e.g., Tesla and Toyota). In this work, we develop an attack against learning-based MDE. In particular, we use an optimization-based method to systematically generate stealthy physical-object-oriented adversarial patches to attack depth estimation. We balance the stealth and effectiveness of our attack with object-oriented adversarial design, sensitive region localization, and natural style camouflage. Using real-world driving scenarios, we evaluate our attack on concurrent MDE models and a representative downstream task for AD (i.e., 3D object detection). Experimental results show that our method can generate stealthy, effective, and robust adversarial patches for different target objects and models and achieves more than 6 m mean depth estimation error and 93% attack success rate (ASR) in object detection with a patch of 1/9 of the vehicle’s rear area. Field tests on three different driving routes with a real vehicle indicate that we cause over 6 m mean depth estimation error and reduce the object detection rate from 90.70% to 5.16% in continuous video frames.
Digital image steganography survey and investigation (goal, assessment, method, development, and dataset)

Digital steganography has a long history, starting to be developed in the 90s until now. The main aspects of early steganography are security, imperceptibility, and payload. Security is an essential aspect of steganography, where its development competes with the steganalysis method. The development of steganographic methods is not only based on statistics but also involves adaptive methods, machine learning, artificial intelligence, generative adversarial networks (GAN), convolution neural networks (CNN), coverless techniques, etc. Several new methods add robustness as an important aspect. This makes the development of steganographic methods more varied and generally focuses the goal of the method on one or two aspects. Nowadays, steganography articles, especially surveys, rarely explicitly classify steganography based on its goals. So this survey contributes to classifying steganographic research based on goals and its assessments. This paper also reviews the use of assessment tools in depth because it is closely related to the goal of steganography. Challenges or issues, methods, developments, and popular datasets are also reviewed. Steganalysis literature was also added to improve the quality of the study and analysis. Finally, this survey provides discussion, investigation, critical analysis, and explicit summary so novice researchers can more easily understand image steganography development.
Autonomous dispatch trajectory planning on flight deck: A search-resampling-optimization framework

There is a growing expectation to realize the autonomous dispatch on flight deck, where dispatch trajectory planning is seen as the key technique. Optimal-control based method has shown great advantages in high degree of constraint satisfaction over its counterparts in the last decade. However, it suffers from low computational efficiency even numerical divergence under scenarios with complicated obstacles. To deal with such an issue, a search-resampling-optimization (SRO) framework is proposed in this paper. A hybrid A* algorithm is employed to generate a coarse path according to the boundary conditions in the search stage. Then a resampling process is implemented to pave a series of safe dispatch corridors (SDCs) along the coarse path. Finally, by replacing the common one-to-one collision-avoidance with the constructed within-SDC constraints, an optimal control problem whose scale is totally independent of the number of obstacles can be formulated. The resampled result is further fed into the optimization stage to facilitate the numerical solution. Dispatch trajectory planning for taxiing aircraft and tractor can be treated uniformly under this framework. And numerical simulations demonstrate that the SRO framework is efficient and robust even with narrow accessible tunnels. The SRO is inherently flexible and can be easily extended to the trajectory planning problem in other fields. A video of the main idea and numerical simulations in this paper is available at www.bilibili.com/video/BV1tP4y1d7xy/.
XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process

Currently, explainability represents a major barrier that Artificial Intelligence (AI) is facing in regard to its practical implementation in various application domains. To combat the lack of understanding of AI-based systems, Explainable AI (XAI) aims to make black-box AI models more transparent and comprehensible for humans. Fortunately, plenty of XAI methods have been introduced to tackle the explainability problem from different perspectives. However, due to the vast search space, it is challenging for ML practitioners and data scientists to start with the development of XAI software and to optimally select the most suitable XAI methods. To tackle this challenge, we introduce XAIR, a novel systematic metareview of the most promising XAI methods and tools. XAIR differentiates itself from existing reviews by aligning its results to the five steps of the software development process, including requirement analysis, design, implementation, evaluation, and deployment. Through this mapping, we aim to create a better understanding of the individual steps of developing XAI software and to foster the creation of real-world AI applications that incorporate explainability. Finally, we conclude with highlighting new directions for future research.
Data Extraction Approach using Natural Language Processing for Sentiment Analysis

The branch of research known as “sentiment analysis and opinion mining” focuses on extracting meaning from the written words of others by studying their thoughts, feelings, judgments, and attitudes. Natural language processing is one of the most active study fields in data mining, web mining, and text mining. There are numerous uses of sentiment analysis, such as assessing the impact of events on social networks and gauging public opinion on products and services. In the same way as blogs, forums, microblogs, and social networks such as Twitter and Facebook have grown in popularity, sentiment analysis is becoming increasingly important. It is possible to measure sentiments that are captured in digital form using supervised machine learning and lexical-based techniques.
EJS: Multi-Strategy Enhanced Jellyfish Search Algorithm for Engineering Applications

The jellyfish search (JS) algorithm impersonates the foraging behavior of jellyfish in the ocean. It is a newly developed metaheuristic algorithm that solves complex and real-world optimization problems. The global exploration capability and robustness of the JS algorithm are strong, but the JS algorithm still has significant development space for solving complex optimization problems with high dimensions and multiple local optima. Therefore, in this study, an enhanced jellyfish search (EJS) algorithm is developed, and three improvements are made: (i) By adding a sine and cosine learning factors strategy, the jellyfish can learn from both random individuals and the best individual during Type B motion in the swarm to enhance optimization capability and accelerate convergence speed. (ii) By adding a local escape operator, the algorithm can skip the trap of local optimization, and thereby, can enhance the exploitation ability of the JS algorithm. (iii) By applying an opposition-based learning and quasi-opposition learning strategy, the population distribution is increased, strengthened, and more diversified, and better individuals are selected from the present and the new opposition solution to participate in the next iteration, which can enhance the solution’s quality, meanwhile, convergence speed is faster and the algorithm’s precision is increased. In addition, the performance of the developed EJS algorithm was compared with those of the incomplete improved algorithms, and some previously outstanding and advanced methods were evaluated on the CEC2019 test set as well as six examples of real engineering cases. The results demonstrate that the EJS algorithm can skip the trap of local optimization, can enhance the solution’s quality, and can increase the calculation speed. In addition, the practical engineering applications of the EJS algorithm also verify its superiority and effectiveness in solving both constrained and unconstrained optimization problems, and therefore, suggests future possible applications for solving such optimization problems.
Natural Language Processing in Electronic Health Records in relation to healthcare decision-making: A systematic review

Natural Language Processing (NLP) is widely used to extract clinical insights from Electronic Health Records (EHRs). However, the lack of annotated data, automated tools, and other challenges hinder the full utilisation of NLP for EHRs. Various Machine Learning (ML), Deep Learning (DL) and NLP techniques are studied and compared to understand the limitations and opportunities in this space comprehensively.After screening 261 articles from 11 databases, we included 127 papers for full-text review covering seven categories of articles: (1) medical note classification, (2) clinical entity recognition, (3) text summarisation, (4) deep learning (DL) and transfer learning architecture, (5) information extraction, (6) Medical language translation and (7) other NLP applications. This study follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.EHR was the most commonly used data type among the selected articles, and the datasets were primarily unstructured. Various ML and DL methods were used, with prediction or classification being the most common application of ML or DL. The most common use cases were: the International Classification of Diseases, Ninth Revision (ICD-9) classification, clinical note analysis, and named entity recognition (NER) for clinical descriptions and research on psychiatric disorders.We find that the adopted ML models were not adequately assessed. In addition, the data imbalance problem is quite important, yet we must find techniques to address this underlining problem. Future studies should address key limitations in studies, primarily identifying Lupus Nephritis, Suicide Attempts, perinatal self-harmed and ICD-9 classification.
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks

In node classification tasks, graph convolutional neural networks (GCNs) have demonstrated competitive performance over traditional methods on diverse graph data. However, it is known that the performance of GCNs degrades with increasing number of layers (oversmoothing problem) and recent studies have also shown that GCNs may perform worse in heterophilous graphs, where neighboring nodes tend to belong to different classes (heterophily problem). These two problems are usually viewed as unrelated, and thus are studied independently, often at the graph filter level from a spectral perspective.We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level. Specifically, we profile the nodes via two quantitative metrics: the relative degree of a node (compared to its neighbors) and the node-level heterophily. Our theory shows that the interplay of these two profiling metrics defines three cases of node behaviors, which explain the oversmoothing and heterophily problems jointly and can predict the performance of GCNs. Based on insights from our theory, we show theoretically and empirically the effectiveness of two strategies: structure-based edge correction, which learns corrected edge weights from structural properties (i.e., degrees), and feature-based edge correction, which learns signed edge weights from node features. Compared to other approaches, which tend to handle well either heterophily or oversmoothing, we show that our model, GGCN, which incorporates the two strategies performs well in both problems. We provide a longer version of this paper in [1] and codes on https://github.com/YujunYan/Heterophily_and_oversmoothing.
Developing a Tuned Three-Layer Perceptron Fed with Trained Deep Convolutional Neural Networks for Cervical Cancer Diagnosis

Cervical cancer is one of the most common types of cancer among women, which has higher death-rate than many other cancer types. The most common way to diagnose cervical cancer is to analyze images of cervical cells, which is performed using Pap smear imaging test. Early and accurate diagnosis can save the lives of many patients and increase the chance of success of treatment methods. Until now, various methods have been proposed to diagnose cervical cancer based on the analysis of Pap smear images. Most of the existing methods can be divided into two groups of methods based on deep learning techniques or machine learning algorithms. In this study, a combination method is presented, whose overall structure is based on a machine learning strategy, where the feature extraction stage is completely separate from the classification stage. However, in the feature extraction stage, deep networks are used. In this paper, a multi-layer perceptron (MLP) neural network fed with deep features is presented. The number of hidden layer neurons is tuned based on four innovative ideas. Additionally, ResNet-34, ResNet-50 and VGG-19 deep networks have been used to feed MLP. In the presented method, the layers related to the classification phase are removed in these two CNN networks, and the outputs feed the MLP after passing through a flatten layer. In order to improve performance, both CNNs are trained on related images using the Adam optimizer. The proposed method has been evaluated on the Herlev benchmark database and has provided 99.23 percent accuracy for the two-classes case and 97.65 percent accuracy for the 7-classes case. The results have shown that the presented method has provided higher accuracy than the baseline networks and many existing methods.
Human activity recognition using marine predators algorithm with deep learning

In the era of smart life, tracking human activities and motion can play a significant role in the advanced modern applications, such as the Internet of things (IoT), Internet of healthcare things (IoHT), smart homes, eldercare, and different health informatics-based applications. Human activity recognition (HAR) has the ability to expose abundant information collected from different devices (i.e., cameras or sensors) that can represent human motion and activities. The recent advances in artificial intelligence methods, including deep learning (DL) and swarm intelligence (SI) optimization algorithms, play a significant role in different applications. In this paper, we integrate the applications of both DL and SI to build a robust HAR system using wearable sensor data. A light feature extraction approach is developed using the residual convolutional network and a recurrent neural network (RCNN-BiGRU). To select the optimal feature set, we develop new feature selection methods based on the marine predator algorithm (MPA). Besides a basic version of the MPA, three binary variants are developed for this goal, called MPAS, MPAS10 and MPAV. We test the proposed MPA variants with comprehensive comparisons to several optimization algorithms using different evaluation indicators as well as statistical tests to ensure their performance quality. We conclude that MPAV recorded the best performance compared to other MPA variants as well as other compared methods.
Ant Colony Optimization-Enabled CNN Deep Learning Technique for Accurate Detection of Cervical Cancer

Cancer is characterized by abnormal cell growth and proliferation, which are both diagnostic indicators of the disease. When cancerous cells enter one organ, there is a risk that they may spread to adjacent tissues and eventually to other organs. Cancer of the cervix of the uterus often initially manifests itself in the uterine cervix, which is located at the very bottom of the uterus. Both the growth and death of cervical cells are characteristic features of this condition. False-negative results provide a significant moral dilemma since they may cause women to get an incorrect diagnosis of cancer, which in turn can result in the woman's premature death from the disease. False-positive results do not raise any significant ethical concerns; but they do require a patient to go through an expensive and time-consuming treatment process, and they also cause the patient to experience tension and anxiety that is not warranted. In order to detect cervical cancer in its earliest stages in women, a screening procedure known as a Pap test is often performed. This article describes a technique for improving images using Brightness Preserving Dynamic Fuzzy Histogram Equalization. To individual components and find the right area of interest, the fuzzy c-means approach is applied. The images are segmented using the fuzzy c-means method to find the right area of interest. The feature selection algorithm is the ACO algorithm. Following that, categorization is carried out utilizing the CNN, MLP, and ANN algorithms.
Fast single-photon detectors and real-time key distillation enable high secret-key-rate quantum key distribution systems

Quantum key distribution has emerged as the most viable scheme to guarantee information security in the presence of large-scale quantum computers and, thanks to the continuous progress made in the past 20 years, it is now commercially available. However, the secret key rates remain limited to just over 10 Mbps due to several bottlenecks on the receiver side. Here we present a custom multipixel superconducting nanowire single-photon detector that is designed to guarantee high count rates and precise timing discrimination. Leveraging the performance of the detector and coupling it to fast acquisition and real-time key distillation electronics, we remove two major roadblocks and achieve a considerable increase of the secret key rates with respect to the state of the art. In combination with a simple 2.5-GHz clocked time-bin quantum key distribution system, we can generate secret keys at a rate of 64 Mbps over a distance of 10.0 km and at a rate of 3.0 Mbps over a distance of 102.4 km with real-time key distillation.
Learners in the Metaverse: A Systematic Review on the Use of Roblox in Learning

The development of the Metaverse has drawn much attention in education. Roblox, as an important platform in the Metaverse, attracts millions of young users, which raises the question of how its effectiveness as a learning environment can be maximized. This study aims to synthesize the available evidence to provide an overview of the current research on learning in Roblox by exploring its benefits, challenges, and existing gaps. In line with PRISMA and assisted by LDA topic modeling, we included 40 studies that were analyzed to answer our questions. The research findings show that: (1) Roblox could be combined with social interactive learning or collaborative learning environments, provide a VR environment that supports learning, and be of benefit to programming in STEM education; (2) the use of Roblox in learning has the advantages of attracting a large number of student users, eliciting the positive attitudes of students, and promoting students’ cognitive and noncognitive learning abilities; and (3) there are also challenges such as cyberbullying, cybersecurity, lack of adequate teaching design, etc. Empirical studies on this topic have only begun to emerge, and more future research is needed into different pedagogical scenarios to explore the effects, factors, outcomes, designs, etc.