Reviewerly/exHarmony huggingface dataset:
Surgical Gesture Recognition in Laparoscopic Tasks Based on the Transformer Network and Self-Supervised Learning

In this study, we propose a deep learning framework and a self-supervision scheme for video-based surgical gesture recognition. The proposed framework is modular. First, a 3D convolutional network extracts feature vectors from video clips for encoding spatial and short-term temporal features. Second, the feature vectors are fed into a transformer network for capturing long-term temporal dependencies. Two main models are proposed, based on the backbone framework: C3DTrans (supervised) and SSC3DTrans (self-supervised). The dataset consisted of 80 videos from two basic laparoscopic tasks: peg transfer (PT) and knot tying (KT). To examine the potential of self-supervision, the models were trained on 60% and 100% of the annotated dataset. In addition, the best-performing model was evaluated on the JIGSAWS robotic surgery dataset. The best model (C3DTrans) achieves an accuracy of 88.0%, a 95.2% clip level, and 97.5% and 97.9% (gesture level), for PT and KT, respectively. The SSC3DTrans performed similar to C3DTrans when training on 60% of the annotated dataset (about 84% and 93% clip-level accuracies for PT and KT, respectively). The performance of C3DTrans on JIGSAWS was close to 76% accuracy, which was similar to or higher than prior techniques based on a single video stream, no additional video training, and online processing.
FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editing

We propose a Few-shot Dynamic Neural Radiance Field (FDNeRF), the first NeRF-based method capable of reconstruction and expression editing of 3D faces based on a small number of dynamic images. Unlike existing dynamic NeRFs that require dense images as input and can only be modeled for a single identity, our method enables face reconstruction across different persons with few-shot inputs. Compared to state-of-the-art few-shot NeRFs designed for modeling static scenes, the proposed FDNeRF accepts view-inconsistent dynamic inputs and supports arbitrary facial expression editing, i.e., producing faces with novel expressions beyond the input ones. To handle the inconsistencies between dynamic inputs, we introduce a well-designed conditional feature warping (CFW) module to perform expression conditioned warping in 2D feature space, which is also identity adaptive and 3D constrained. As a result, features of different expressions are transformed into the target ones. We then construct a radiance field based on these view-consistent features and use volumetric rendering to synthesize novel views of the modeled faces. Extensive experiments with quantitative and qualitative evaluation demonstrate that our method outperforms existing dynamic and few-shot NeRFs on both 3D face reconstruction and expression editing tasks. Code is available at https://fdnerf.github.io .
FloRen: Real-time High-quality Human Performance Rendering via Appearance Flow Using Sparse RGB Cameras

We propose FloRen, a novel system for real-time, high-resolution free-view human synthesis. Our system runs at 15fps in 1K resolution with very sparse RGB cameras. In FloRen, a coarse-level implicit geometry is recovered at first as initialization, and then processed by a neural rendering framework based on appearance flow. Our appearance flow-based rendering framework consists of three steps, namely view-dependent depth refinement, appearance flow estimation and occlusion-aware color rendering. In this way, we resolve the view synthesis problem in the image plane, where 2D convolutional neural networks can be efficiently applied, contributing to high speed performance. For robust appearance flow estimation, we explicitly combine data-driven human prior knowledge with multiview geometric constraints. The accurate appearance flow enables precise color mapping from input view to novel view, which greatly facilitates high-resolution novel view generation. We demonstrate that our system achieves state-of-the-art performance and even outperforms many offline methods.
Adaptive path planning method for UAVs in complex environments

Path planning is an important problem in the field of unmanned aerial vehicles (UAVs), particularly in complex environments; however, existing path planning methods have certain limitations and yield poor results. For a better solution to the path planning problem, we propose an adaptive path planning method for UAVs in complex environments. This method is based on discrete global grid systems for conflict detection between the airspace and UAV path. A multi-scale discrete layered grid model that provides a new management framework for the discrete global grid and accelerates conflict detection is proposed for complex environments. Thereafter, the particle swarm optimization (PSO) was exploited to develop an adaptive path planning PSO (APP-PSO) method, which was improved in terms of dimension, initialization, and iteration update strategy to plan an optimal path. Finally, the proposed method was validated by comparison with other related PSO algorithms and several simulation-based experiments illustrating its optimality.
Ensemble and Self-supervised Learning for Improved Classification of Seismic Signals from the Åknes Rockslope

Abstract A case study with seismic geophone data from the unstable Åknes rock slope in Norway is considered. This rock slope is monitored because there is a risk of severe flooding if the massive-size rock falls into the fjord. The geophone data is highly valuable because it provides 1000 Hz sampling rates data which are streamed to a web resource for real-time analysis. The focus here is on building a classifier for these data to distinguish different types of microseismic events which are in turn indicative of the various processes occurring on the slope. There are 24 time series from eight 3-component geophone data for about 3500 events in total, and each of the event time series has a length of 16 s. For the classification task, novel machine learning methods such as deep convolutional neural networks are leveraged. Ensemble prediction is used to extract information from all time series, and this is seen to give large improvements compared with doing immediate aggregation of the data. Further, self-supervised learning is evaluated to give added value here, in particular for the case with very limited training data.
Educational Innovation in Adult Learning Considering Digital Transformation for Social Inclusion

This research seeks to identify teaching-learning mechanisms that allow innovation and educational evolution for Youth and Adult Basic Education and thus achieve social inclusion in a rural development environment that has technological limitations and internet access in rural areas. The work seeks to identify a strategic process of learning objects to apply an inverted classroom in no presential modality. The objective is to respond to the low level of knowledge in the area of Language and Literature to mitigate the lack of understanding of the andragogical resources of the teachers in virtual classes. A methodological strategy is proposed that is related to the scientific field through bibliometric and quantitative analysis based on scientific information; in a second moment, the environment is evaluated via satisfaction surveys conducted with students and teachers of the third baccalaureate of the rural areas in the no presential modality of the Juan Jiménez Educational Unit, Abdón Calderón extension, province of Sucumbíos, Ecuador. A total of 66.67% of teachers between 41 and 45 agree with implementing ICT as a strategy for better learning in the education of young people and adults despite the scarce access to the internet in rural areas.
AMSA: Adaptive Multimodal Learning for Sentiment Analysis

Efficient recognition of emotions has attracted extensive research interest, which makes new applications in many fields possible, such as human-computer interaction, disease diagnosis, service robots, and so forth. Although existing work on sentiment analysis relying on sensors or unimodal methods performs well for simple contexts like business recommendation and facial expression recognition, it does far below expectations for complex scenes, such as sarcasm, disdain, and metaphors. In this article, we propose a novel two-stage multimodal learning framework, called AMSA, to adaptively learn correlation and complementarity between modalities for dynamic fusion, achieving more stable and precise sentiment analysis results. Specifically, a multiscale attention model with a slice positioning scheme is proposed to get stable quintuplets of sentiment in images, texts, and speeches in the first stage. Then a Transformer-based self-adaptive network is proposed to assign weights flexibly for multimodal fusion in the second stage and update the parameters of the loss function through compensation iteration. To quickly locate key areas for efficient affective computing, a patch-based selection scheme is proposed to iteratively remove redundant information through a novel loss function before fusion. Extensive experiments have been conducted on both machine weakly labeled and manually annotated datasets of self-made Video-SA, CMU-MOSEI, and CMU-MOSI. The results demonstrate the superiority of our approach through comparison with baselines.
Estimating small structural motions based on sparsity enforcement

Abstract This paper proposed a novel, efficient method to estimate challenging small structural motions from noisy video. To eliminate the phase limitation, ill‐posed problem, and high computational burden, the structural motion function is resampled and recovered. Because video signals have tremendous redundancies in spatial, block, and time domains, the objective is achieved by enforcing three levels of sparsity constraints. In the first step, the ill‐posed optimization is solved using the multi‐hypothesis prediction embedded with the Tikhonov regularization, to resample and recover the video signal in the spatial domain. The second step is to characterize the measurement uncertainty in the block domain, in which two weight matrices are proposed into the regularization terms of the sparse coding. Finally, the temporal correlation of video frames is exploited by the reweighted residual sparsity. The superiority of the sparsity‐enforcement method over existing methods was demonstrated through several case studies. Among the comparisons, the sparsity‐enforcement method yielded video magnifications, structural motions, and modal information more accurately. Meanwhile, it has the lowest computational burden.
Toward Holistic Scene Understanding: A Transfer of Human Scene Perception to Mobile Robots

The long-term vision for robotics is to have fully autonomous mobile robots that perceive the environment as humans do or even better. This article transfers the core ideas from human scene perception onto robot scene perception to contribute toward a holistic scene understanding of robots. The first contribution is to extensively survey and compare state-of-the-art robot scene perception approaches with neuroscience theories and studies of human perception. A step-by-step transfer of the perceptual process reveals similarities and differences between robots and humans. The second contribution represents an analysis of the status quo of holistic robot perception approaches to extract to what extent the perceptual capabilities of humans have been reached. Building on this, the gaps and potentials of robot perception are illustrated to address future research directions.
Divergence-Driven Consistency Training for Semi-Supervised Facial Age Estimation

Facial age estimation has attracted considerable attention owing to its great potential in applications. However, it still falls short of reliable age estimation due to the lack of sufficient training data with accurate age labels. Using conventional semi-supervised methods to exploit unlabeled data appears to be a good solution, but it does not yield sufficient performance gains while significantly increasing training time. Therefore, to tackle these problems, we present a Divergence-driven Consistency Training (DCT) method for enhancing both efficiency and performance in this paper. Following the idea of pseudo-labeling and consistency regularization, we assign pseudo labels predicted by the teacher model to unlabeled samples and then train the student model on labeled and unlabeled samples based on consistency regularization. Based on this, we propose two main promotions. The first is the Efficient Sample Selection (ESS) strategy, which is based on the Divergence Score to select effective samples from massive unlabeled images to reduce the training time and improve efficiency. The second is Identity Consistency (IC) regularization as the additional loss function, which introduces a high dependency of aging traits on a person. Moreover, we propose Local Prediction (LP), which is a plug-and-play component, to capture local semantics. Extensive experiments on multiple age benchmark datasets, including CACD, Morph II, MIVIA, and Chalearn LAP 2015, indicate DCT outperforms the state-of-the-art approaches significantly.
SWDet: Anchor-Based Object Detector for Solid Waste Detection in Aerial Images

As we all know, waste pollution is one of the most serious environmental issues in the world. Efficient detection of solid waste (SW) in aerial images can improve subsequent waste classification and automatic sorting on the ground. However, traditional methods have some problems, such as poor generalization and limited detection performance. This article presents an anchor-based object detector for solid waste in aerial images (SWDet). Specifically, we construct asymmetric deep aggregation (ADA) network with structurally reparameterized asymmetric blocks to extract waste features with inconspicuous appearance. Besides, considering the waste with blurred boundaries caused by the resolution of aerial images, this article constructs efficient attention fusion pyramid network (EAFPN) to obtain contextual information and multiscale geospatial information via attention fusion. And the model can capture the scattering features of irregular shape waste. In addition, we construct the dataset for solid waste aerial detection (SWAD) by collecting aerial images of SW in Henan Province, China, to validate the effectiveness of our method. Experimental results show that SWDet outperforms most of existing methods for SW detection in aerial images.
ARFED: Attack-Resistant Federated averaging based on outlier elimination

In federated learning, each participant trains its local model with its own data and a global model is formed at a trusted server by aggregating model updates coming from these participants. Since the server has no effect and visibility on the training procedure of the participants to ensure privacy, the global model becomes vulnerable to attacks such as data poisoning and model poisoning. Although many defense algorithms have recently been proposed to address these attacks, they often make strong assumptions that do not agree with the nature of federated learning, such as assuming Non-IID datasets. Moreover, they mostly lack comprehensive experimental analyses. In this work, we propose a defense algorithm called ARFED that does not make any assumptions about data distribution, update similarity of participants, or the ratio of the malicious participants. ARFED mainly considers the outlier status of participant updates for each layer of the model architecture based on the distance to the global model. Hence, the participants that do not have any outlier layer are involved in model aggregation. We have performed extensive experiments on diverse scenarios and shown that the proposed approach provides a robust defense against different attacks. To test the defense capability of the ARFED in different conditions, we considered label flipping, Byzantine, and partial knowledge attacks for both IID and Non-IID settings in our experimental evaluations. Moreover, we proposed a new attack, called organized partial knowledge attack, where malicious participants use their training statistics collaboratively to define a common poisoned model. We have shown that organized partial knowledge attacks are more effective than independent attacks.
Shortcomings and areas for improvement in digital pathology image segmentation challenges

Digital pathology image analysis challenges have been organised regularly since 2010, often with events hosted at major conferences and results published in high-impact journals. These challenges mobilise a lot of energy from organisers, participants, and expert annotators (especially for image segmentation challenges). This study reviews image segmentation challenges in digital pathology and the top-ranked methods, with a particular focus on how reference annotations are generated and how the methods' predictions are evaluated. We found important shortcomings in the handling of inter-expert disagreement and the relevance of the evaluation process chosen. We also noted key problems with the quality control of various challenge elements that can lead to uncertainties in the published results. Our findings show the importance of greatly increasing transparency in the reporting of challenge results, and the need to make publicly available the evaluation codes, test set annotations and participants' predictions. The aim is to properly ensure the reproducibility and interpretation of the results and to increase the potential for exploitation of the substantial work done in these challenges.
K-12 educators’ self-confidence in designing and implementing cybersecurity lessons

Cybersecurity awareness and education initiatives are being designed and expanded for the workforce and K-20 education institutions. Predominately, K-12 cybersecurity education programs and research have focused on student learning experiences; however, there is a need to also concentrate cybersecurity education resources on K-12 teacher professional development. Teacher professional development supporting self-confidence in knowledge building and modeling of instructional practices may positively influence K-12 teachers' integration of cybersecurity concepts within classroom instruction. This study documented K-12 teachers' (n=17) perceived self-confidence in designing and implementing cybersecurity-related lessons when participating in a week-long cybersecurity education workshop experience funded by the National Security Agency's GenCyber grant program called the [Redacted Program Name]. Teachers were asked to rate their perceived self-confidence (pre and post) related to cybersecurity lesson development. Results showed that all self-confidence survey items increased after engaging in the GenCyber workshop; however, two items focused on creating engaging, inquiry-based lessons were not significant at the alpha value level of 0.003. These findings suggest that effective professional development aimed to increase K-12 teachers' cybersecurity knowledge and to support development and implementation of cybersecurity-related lesson plans is an important factor in advocating for cybersecurity education efforts in K-12 education.
Pedestrian Detection for Autonomous Cars: Inference Fusion of Deep Neural Networks

Network fusion has been recently explored as an approach for improving pedestrian detection performance. However, most existing fusion methods suffer from runtime efficiency, modularity, scalability, and maintainability due to the complex structure of the entire fused models, their end-to-end training requirements, and sequential fusion process. Addressing these challenges, this paper proposes a novel fusion framework that combines asymmetric inferences from object detectors and semantic segmentation networks for jointly detecting multiple pedestrians. This is achieved by introducing a consensus-based scoring method that fuses pair-wise pixel-relevant information from the object detector and the semantic segmentation network to boost the final confidence scores. The parallel implementation of the object detection and semantic segmentation networks in the proposed framework entails a low runtime overhead. The efficiency and robustness of the proposed fusion framework are extensively evaluated by fusing different state-of-the-art pedestrian detectors and semantic segmentation networks on a public dataset. The generalization of fused models is also examined on new cross pedestrian data collected through an autonomous car. Results show that the proposed fusion method significantly improves detection performance while achieving competitive runtime efficiency.
Momentum-Based Variance-Reduced Proximal Stochastic Gradient Method for Composite Nonconvex Stochastic Optimization

Stochastic gradient methods (SGMs) have been extensively used for solving stochastic problems or large-scale machine learning problems. Recent works employ various techniques to improve the convergence rate of SGMs for both convex and nonconvex cases. Most of them require a large number of samples in some or all iterations of the improved SGMs. In this paper, we propose a new SGM, named PStorm, for solving nonconvex nonsmooth stochastic problems. With a momentum-based variance reduction technique, PStorm can achieve the optimal complexity result $$O(\varepsilon ^{-3})$$ to produce a stochastic $$\varepsilon $$ -stationary solution, if a mean-squared smoothness condition holds. Different from existing optimal methods, PStorm can achieve the $${O}(\varepsilon ^{-3})$$ result by using only one or O(1) samples in every update. With this property, PStorm can be applied to online learning problems that favor real-time decisions based on one or O(1) new observations. In addition, for large-scale machine learning problems, PStorm can generalize better by small-batch training than other optimal methods that require large-batch training and the vanilla SGM, as we demonstrate on training a sparse fully-connected neural network and a sparse convolutional neural network.
A machine learning based framework to identify unseen classes in open-world text classification

Classical supervised machine learning (ML) follows the assumptions of closed-world learning. However, this assumption does not work in an open-world dynamic environment. Therefore, the automated systems must be able to discover and identify unseen instances. Open-world ML can deal with unseen instances and classes through a two-step process: (1) discover and classify unseen instances and (2) identify novel classes discovered in step (1). Most existing research on open-world machine learning (OWML) only focuses on step 1. However, performing step 2 is required to build intelligent systems. The proposed framework comprises three different but interconnected modules that discover and identify unseen classes. Our in-depth performance evaluation establishes that the proposed framework improves open accuracy by up to 8% compared to the state-of-the-art models.