Reviewerly/exHarmony huggingface dataset:
Truncated attention-aware proposal networks with multi-scale dilation for temporal action detection

Detecting actions temporally in untrimmed videos is very challenging, and it accomplishes action classification and localization simultaneously. Capturing the relations among action proposals (i.e., candidate video segments) is of vital importance. While there have been several attempts to encode such relations, they neglect the adverse effects of those irrelevant or negative relations among proposals. Besides, there is a crucial fact that action durations are flexible in videos, which has not been well explored. For the former, we develop a truncated attention mechanism that learns positive proposal relations by dynamically adjusting edge weights of proposal nodes in a graph, and construct the proposal network model using graph convolution networks to suppress disadvantageous relations of proposal pairs by truncating negative attention scores. For the latter, we devise a light multi-scale dilation module shared by all proposals to handle different action durations by enlarging temporal receptive field, thus capturing temporal context to increase the representation capacity of proposals. Unifying these considerations, we present the Multi-scale Dilation based Truncated Attention Proposal Network (MD-TAPN) model for temporal action detection. Our model achieves state-of-the-art performances of detecting actions on two benchmark databases, and especially it outperforms the most competitive method by a significant gain of 3.6% mAP at tIoU0.5 on THUMOS14.
A deep multi-embedding model for mobile application recommendation

With the explosion in the number of mobile applications, it becomes difficult and time-consuming for users to find the most suitable and interesting application from the millions of applications that exist today. Therefore, mobile app recommendation becomes a top priority to help people solve this problem. In previous studies, there are two common methods to solve this problem, one is to apply matrix factorization to the rating data to predict the target user's ratings for other applications, and the other is to predict ratings based on the user's review text. In addition to combining the above two traditional methods, this study also combines deep learning methods to generate better models. For rating data, we use a deep matrix factorization method to obtain latent factor vectors of users and items from the rating matrix to improve the embedding of ratings. For review text, we improve the text representation by using a multi-embedding approach. Most importantly, our model also incorporates deep neural networks with nonlinear transformation capabilities to extract more representative abstract features. We conduct experiments on datasets collected from the real world. Extensive experimental results show that our proposed method has better performance than other existing methods.
Alleviating the generalization issue in adversarial domain adaptation networks

Recently, adversarial learning has dominated domain adaptation, which is a popular branch of transfer learning. The basic idea of adversarial domain adaptation networks (ADAN) is to learn domain-invariant features which is able to confuse the domain discriminator. By sharing the spirit of generative adversarial networks (GANs), ADAN achieved state-of-the-art performance. However, ADAN also inherits the drawbacks of GANs. One of the most critical issues of GANs is that the learned distribution may be far from the expected one even if the training is successful, which is known as the generalization issue in GANs. As a result, it is no guarantee that the learned representations are domain-invariant even if the domain discriminator is successfully confused. To address this, we propose a new domain adaptation approach under the framework of ADAN. Specifically, we reformulate the conventional ADAN and propose ADAN plus metric protocol under the new ADAN formulation, ADANM for short, which leverages both adversarial learning and metric learning. The proposed method, on one hand, challenges the generalization issue in previous ADAN approaches. On the other hand, it guarantees that domain divergence is minimized during the adversarial training. Extensive experiments on three public benchmarks verify that the proposed protocol is favorable for unsupervised domain adaptation tasks.
College English teaching reform and innovative methods under the new media platform based on the IoT

Hybrid teaching has emerged along with the growth of web technology. Teaching activities are combined with online and offline teaching. Some are online teaching and some are offline teaching. In addition to completing classroom teaching, teachers also need to help students answer questions and learn interactively online. The implementation of these links needs to be reflected in the teacher’s teaching evaluation. The article adopted the analytic hierarchy process, which enabled teachers (managers) to more objectively select and determine the evaluation factors, and decompose them at different levels to make quantitative evaluation, so as to provide appropriate information for teachers (managers) and reduce decision-making errors. In this paper, the effectiveness of the new media English teaching platform in colleges and universities in mainland China was evaluated by the method of final test, and all students signed the informed consent. According to different organizational structures, the new media English teaching platform in universities can be divided into user layer, terminal layer, application layer, platform layer and network layer. Each layer is connected with each other, which organizes all components of the new media English teaching platform of the whole university. The network layer is an essential part of college new media English teaching, including the Internet, Internet of Things (IoT), campus network, education network, etc. Through the Internet, educational network and campus network, various educational institutions and institutions were connected, and information exchange was carried out through the university’s new media English platform. The emergence of the IoT and mobile communication networks has greatly improved the network coverage and the utilization rate of the network. The average students' test score using the proposed platform (80.6) is higher than the average test score without the platform (69.1).The new media teaching platform proposed in the article has enabled teachers to effectively complete the teaching content and effectively enhance the teaching quality.
Robust audio copy-move forgery detection on short forged slices using sliding window

Audio copy-move forgery detection (CMFD), which aims at finding possible forgeries that are derived from the same audio recording, has been one of the most difficult challenges in blind audio forensics. However, the existing works rarely pay attention to the forgery of short slices within voiced segments, which may result in missed detection. In this work, we present a robust audio CMFD method on the basis of sliding window (SW) and constant Q cepstral coefficients (CQCC). Specifically, we first propose two SW strategies to illustrate how to reveal the possible copy-move forgeries within and between voiced segments. Then, we combine the CQCC feature with the Pearson correlation coefficients to measure the similarity between different short slices. Finally, we evaluate the proposed method, named SW-CQCC, against the state-of-the-art approaches to CMFD on real-world read English and Chinese corpora. Experimental results demonstrate that our SW-CQCC exhibits significantly high effectiveness and robustness in detecting short forged slices within voiced segments, which may be helpful to audio forensic examinations.
Robustness Implies Privacy in Statistical Estimation

We study the relationship between adversarial robustness and differential privacy in high-dimensional algorithmic statistics. We give the first black-box reduction from privacy to robustness which can produce private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy for a wide range of fundamental high-dimensional parameter estimation problems, including mean and covariance estimation. We show that this reduction can be implemented in polynomial time in some important special cases. In particular, using nearly-optimal polynomial-time robust estimators for the mean and covariance of high-dimensional Gaussians which are based on the Sum-of-Squares method, we design the first polynomial-time private estimators for these problems with nearly-optimal samples-accuracy-privacy tradeoffs. Our algorithms are also robust to a nearly optimal fraction of adversarially-corrupted samples.
Blockchain Transaction Fee Forecasting: A Comparison of Machine Learning Methods

Gas is the transaction-fee metering system of the Ethereum network. Users of the network are required to select a gas price for submission with their transaction, creating a risk of overpaying or delayed/unprocessed transactions in this selection. In this work, we investigate data in the aftermath of the London Hard Fork and shed insight into the transaction dynamics of the net-work after this major fork. As such, this paper provides an update on work previous to 2019 on the link between EthUSD BitUSD and gas price. For forecasting, we compare a novel combination of machine learning methods such as Direct Recursive Hybrid LSTM, CNNLSTM, and Attention LSTM. These are combined with wavelet threshold denoising and matrix profile data processing toward the forecasting of block minimum gas price, on a 5-min timescale, over multiple lookaheads. As the first application of the matrix profile being applied to gas price data and forecasting we are aware of, this study demonstrates that matrix profile data can enhance attention-based models however, given the hardware constraints, hybrid models outperformed attention and CNNLSTM models. The wavelet coherence of inputs demonstrates correlation in multiple variables on a 1 day timescale, which is a deviation of base free from gas price. A Direct-Recursive Hybrid LSTM strategy outperforms other models. Hybrid models have favourable performance up to a 20 min lookahead with performance being comparable to attention models when forecasting 25/50-min ahead. Forecasts over a range of lookaheads allow users to make an informed decision on gas price selection and the optimal window to submit their transaction in without fear of their transaction being rejected. This, in turn, gives more detailed insight into gas price dynamics than existing recommenders, oracles and forecasting approaches, which provide simple heuristics or limited lookahead horizons.
MAMIQA: No-Reference Image Quality Assessment Based on Multiscale Attention Mechanism With Natural Scene Statistics

No-Reference Image Quality Assessment aims to evaluate the perceptual quality of an image, according to human perception. Many recent studies use Transformers to assign different self-attention mechanisms to distinguish regions of an image, simulating the perception of the human visual system (HVS). However, the quadratic computational complexity caused by the self-attention mechanism is time-consuming and expensive. Meanwhile, the image resizing in the feature extraction stage loses the full-size image quality. To address these issues, we propose a lightweight attention mechanism using decomposed large-kernel convolutions to extract multiscale features, and a novel feature enhancement module to simulate HVS. We also propose to compensate the information loss caused by image resizing, with supplementary features from natural scene statistics. Experimental results on five standard datasets show that the proposed method surpasses the SOTA, while significantly reducing the computational costs.
An End-to-End Energy-Efficient Approach for Intake Detection With Low Inference Time Using Wrist-Worn Sensor

Automated detection of intake gestures with wearable sensors has been a critical area of research for advancing our understanding and ability to intervene in people's eating behavior. Numerous algorithms have been developed and evaluated in terms of accuracy. However, ensuring the system is not only accurate in making predictions but also efficient in doing so is critical for real-world deployment. Despite the growing research on accurate detection of intake gestures using wearables, many of these algorithms are often energy inefficient, impeding on-device deployment for continuous and real-time monitoring of diet. This article presents a template-based optimized multicenter classifier that enables accurate intake gesture detection while maintaining low-inference time and energy consumption using a wrist-worn accelerometer and gyroscope. We designed an Intake Gesture Counter smartphone application (CountING) and validated the practicality of our algorithm against seven state-of-the-art approaches on three public datasets (In-lab FIC, Clemson, and OREBA). Compared with other methods, we achieved optimal accuracy (81.60% F1 score) and very low inference time (15.97 msec per 2.20-sec data sample) on the Clemson dataset, and among the top performing algorithms, we achieve comparable accuracy (83.0% F1 score compared with 85.6% in the top performing algorithm) but superior inference time (13.8x faster, 33.14 msec per 2.20-sec data sample) on the In-lab FIC dataset and comparable accuracy (83.40% F1 score compared with 88.10% in the top-performing algorithm) but superior inference time (33.9x faster, 16.71 msec inference time per 2.20-sec data sample) on the OREBA dataset. On average, our approach achieved a 25-hour battery lifetime (44% to 52% improvement over state-of-the-art approaches) when tested on a commercial smartwatch for continuous real-time detection. Our approach demonstrates an effective and efficient method, enabling real-time intake gesture detection using wrist-worn devices in longitudinal studies.
Adaptive Part Mining for Robust Visual Tracking

Visual tracking aims to estimate object state in a video sequence, which is challenging when facing drastic appearance changes. Most existing trackers conduct tracking with divided parts to handle appearance variations. However, these trackers commonly divide target objects into regular patches by a hand-designed splitting way, which is too coarse to align object parts well. Besides, a fixed part detector is difficult to partition targets with arbitrary categories and deformations. To address the above issues, we propose a novel adaptive part mining tracker (APMT) for robust tracking via a transformer architecture, including an object representation encoder, an adaptive part mining decoder, and an object state estimation decoder. The proposed APMT enjoys several merits. First, in the object representation encoder, object representation is learned by distinguishing target object from background regions. Second, in the adaptive part mining decoder, we introduce multiple part prototypes to adaptively capture target parts through cross-attention mechanisms for arbitrary categories and deformations. Third, in the object state estimation decoder, we propose two novel strategies to effectively handle appearance variations and distractors. Extensive experimental results demonstrate that our APMT achieves promising results with high FPS. Notably, our tracker is ranked the first place in the VOT-STb2022 challenge.
Tracking of Elliptical Object with Unknown but Fixed Lengths of Axes

This paper addresses the problem of tracking an elliptical object (e.g., a vehicle or aircraft carrier) with unknown but fixed lengths of axes. In practice, such axis lengths are usually time invariant, but the orientation and kinematics may be time varying. To model this extended object tracking (EOT) problem well, we represent the kinematics and orientation by a random vector, and represent the axis lengths by non-random unknown parameters. We investigate the expectation-maximization (EM) algorithm and propose an EM-based EOT approach, which utilizes the prior information about the invariant lengths and estimates the state and parameters in a unified framework. To reduce computation for real-time applications, we develop a recursive, easy to implement approach. Handy and efficient estimation of axis lengths is developed. Simulation and real-data results are presented to illustrate the effectiveness of our modeling and approach.
Manifold-Contrastive Broad Learning System for Wheelset Bearing Fault Diagnosis

Newly deployed trains have massive normal data and scarce faulty data for training, which limits the diagnosis accuracy with class imbalance problem of small samples. Considering that there are a lot unutilized information hidden in the abundant unlabeled monitoring data, this paper proposes a novel method named manifold-contrastive broad learning system, which utilizes the online updating approach for dealing with the class imbalance problem of small samples. This method constructs a novel one-class broad-learning classifier based on an inherency-guided comparison mechanism, which can classify and annotate unlabeled data online. This classifier employs contrastive manifold matrices to maintain the inherent structures, which is not affected to the overfitting caused by imbalanced samples. Secondly, inspired by the active learning, this classifier proposes the minimum-error strategy to annotate the samples by classifying the modes, which solves the problem of insufficient training data. Thirdly, this method applies an incremental learning strategy that continuously absorbs the newly annotated data to update the model online, which improves the model accuracy under the data imbalanced condition. Finally, the feasibility and effectiveness of the proposed method are verified by wheelset bearing data collected from a test rig of a Chinese rolling stock company.
Multilayer Evolving Fuzzy Neural Networks

It is widely recognized that learning systems have to go deeper to exchange for more powerful representational learning capabilities in order to precisely approximate nonlinear complex problems. However, the best-known computational intelligence approaches with such characteristics, namely, deep neural networks, are often criticized for lacking transparency. In this article, a novel multilayer evolving fuzzy neural network (MEFNN) with a transparent system structure is proposed. The proposed MEFNN is a metalevel stacking ensemble learning system composed of multiple cascading evolving neuro-fuzzy inference systems (ENFISs), processing input data layer-by-layer to automatically learn multilevel nonlinear distributed representations from data. Each ENFIS is an evolving fuzzy system capable of learning from new data sample by sample to self-organize a set of human-interpretable IF– THEN fuzzy rules that facilitate approximate reasoning. Adopting ENFIS as its ensemble component, the multilayer system structure of the MEFNN is flexible and transparent, and its internal reasoning and decision-making mechanism can be explained and interpreted to/by humans. To facilitate information exchange between different layers and attain stronger representation learning capability, the MEFNN utilizes error backpropagation to self-update the consequent parameters of the IF–THEN rules of each ensemble component based on the approximation error propagated backward. To enhance the capability of the MEFNN to handle complex problems, a nonlinear activation function is introduced to modeling the consequent parts of the IF–THEN rules of ENFISs, thereby empowering both the representation and the reflection of nonlinearity in the resulting fuzzy outputs. Numerical examples on a wide variety of challenging (benchmark and real-world) classification and regression problems demonstrate the superior practical performance of the MEFNN, revealing the effectiveness and validity of the proposed approach.
Deep Learning-Based Sentiment Analysis for Social Media

Due to the continuous popularization of the Internet and mobile phones, people have gradually entered a participatory network era, and the rapid growth of social networks has caused an explosion of digital information content. It has turned online opinions, blogs, tweets and posts into highly valuable assets, allowing governments and businesses to gain insights from the data and make their strategies. Business organizations need to process and analyze these sentiments to investigate the data and gain business insights. In recent years, deep learning techniques have been very successful in performing sentiment analysis, which offers automatic feature extraction, rich representation capabilities and better performance compared with traditional feature-based techniques. The core idea is to extract complex features automatically from large amounts of data by building deep neural networks to generate up-to-date predictions. This paper reviews social media sentiment analysis methods based on deep learning. Firstly, it introduces the process of single-modal text sentiment analysis on social media. Then it summarizes the multimodal sentiment analysis algorithms for social media, and divides the algorithm into feature layer fusion, decision layer fusion and linear regression model according to different fusion strategies. finally, the difficulties of social media sentiment analysis based on deep learning and future research directions are discussed.
A Chinese Named Entity Recognition Method Fusing Word and Radical Features

Named Entity Recognition (NER) is a subtask of natural language processing. Its accuracy is crucial for downstream tasks. In Chinese NER, word information is often added to enhance the semantic and boundary information of Chinese words, but these methods ignore the radical information of Chinese characters. This paper propose a multi-feature fusion model(MFFM) for Chinese NER. First, the input sequences are exported to the BERT layer, the word embedding layer and the radical embedding layer respectively; then the above three layer output are combined together as input of the Bidirectional Long Short-Term Memory(BiLSTM) layer to model the contextual information; finally annotate the sequence with conditional random field. The proposed model not only avoids the import of complex structures, but also effectively captures the character features of the context, thus improves the recognition performance. The experimental results show that the F1 value of MFFM reaches 71.02% on the Weibo dataset, which is 3.12% higher than that of the BERT model, and 82.78% on the OntoNotes4.0 dataset, which is 0.85% higher than that of the BERT model.
Transfer learning for ensembles: reducing computation time and keeping the diversity

Transferring a deep neural network trained on one problem to another requires only a small amount of data and little additional computation time. The same behaviour holds for ensembles of deep learning models typically superior to a single model. However, a transfer of deep neural networks ensemble demands relatively high computational expenses. The probability of overfitting also increases.
The Use of Metaverse in the Healthcare Sector: Analysis and Applications

With the use of blockchain, Internet of Things, virtual platform/telecommunications network, artificial intelligence and the fourth industrial revolution, the essential demand for digital transition within the health care settings has increased as an outcome of the 2019 coronavirus illness outbreak and the fourth industrial revolution. The evolution of virtual environments with three-dimensional (3D) spaces and avatars, known as metaverse, has slowly gained acceptance in the field of health care. These environments may be especially useful for patient-facing platforms (such as platforms for telemedicine), functional uses (such as meeting management), digital education (such as modeled medical and surgical learning), treatments and diagnoses. This chapter offers the most recent state-of-the-art metaverse services and applications and a growing problem when it comes to using it in the healthcare sector.