Reviewerly/exHarmony huggingface dataset:
NPU RGBD Dataset and a Feature-Enhanced LSTM-DGCN Method for Action Recognition of Basketball Players+

Computer vision-based action recognition of basketball players in basketball training and competition has gradually become a research hotspot. However, owing to the complex technical action, diverse background, and limb occlusion, it remains a challenging task without effective solutions or public dataset benchmarks. In this study, we defined 32 kinds of atomic actions covering most of the complex actions for basketball players and built the dataset NPU RGB+D (a large scale dataset of basketball action recognition with RGB image data and Depth data captured in Northwestern Polytechnical University) for 12 kinds of actions of 10 professional basketball players with 2169 RGB+D videos and 75 thousand frames, including RGB frame sequences, depth maps, and skeleton coordinates. Through extracting the spatial features of the distances and angles between the joint points of basketball players, we created a new feature-enhanced skeleton-based method called LSTM-DGCN for basketball player action recognition based on the deep graph convolutional network (DGCN) and long short-term memory (LSTM) methods. Many advanced action recognition methods were evaluated on our dataset and compared with our proposed method. The experimental results show that the NPU RGB+D dataset is very competitive with the current action recognition algorithms and that our LSTM-DGCN outperforms the state-of-the-art action recognition methods in various evaluation criteria on our dataset. Our action classifications and this NPU RGB+D dataset are valuable for basketball player action recognition techniques. The feature-enhanced LSTM-DGCN has a more accurate action recognition effect, which improves the motion expression ability of the skeleton data.
Predicting Semantic Similarity Between Clinical Sentence Pairs Using Transformer Models: Evaluation and Representational Analysis

Background Semantic textual similarity (STS) is a natural language processing (NLP) task that involves assigning a similarity score to 2 snippets of text based on their meaning. This task is particularly difficult in the domain of clinical text, which often features specialized language and the frequent use of abbreviations. Objective We created an NLP system to predict similarity scores for sentence pairs as part of the Clinical Semantic Textual Similarity track in the 2019 n2c2/OHNLP Shared Task on Challenges in Natural Language Processing for Clinical Data. We subsequently sought to analyze the intermediary token vectors extracted from our models while processing a pair of clinical sentences to identify where and how representations of semantic similarity are built in transformer models. Methods Given a clinical sentence pair, we take the average predicted similarity score across several independently fine-tuned transformers. In our model analysis we investigated the relationship between the final model’s loss and surface features of the sentence pairs and assessed the decodability and representational similarity of the token vectors generated by each model. Results Our model achieved a correlation of 0.87 with the ground-truth similarity score, reaching 6th place out of 33 teams (with a first-place score of 0.90). In detailed qualitative and quantitative analyses of the model’s loss, we identified the system’s failure to correctly model semantic similarity when both sentence pairs contain details of medical prescriptions, as well as its general tendency to overpredict semantic similarity given significant token overlap. The token vector analysis revealed divergent representational strategies for predicting textual similarity between bidirectional encoder representations from transformers (BERT)–style models and XLNet. We also found that a large amount information relevant to predicting STS can be captured using a combination of a classification token and the cosine distance between sentence-pair representations in the first layer of a transformer model that did not produce the best predictions on the test set. Conclusions We designed and trained a system that uses state-of-the-art NLP models to achieve very competitive results on a new clinical STS data set. As our approach uses no hand-crafted rules, it serves as a strong deep learning baseline for this task. Our key contribution is a detailed analysis of the model’s outputs and an investigation of the heuristic biases learned by transformer models. We suggest future improvements based on these findings. In our representational analysis we explore how different transformer models converge or diverge in their representation of semantic signals as the tokens of the sentences are augmented by successive layers. This analysis sheds light on how these “black box” models integrate semantic similarity information in intermediate layers, and points to new research directions in model distillation and sentence embedding extraction for applications in clinical NLP.
Are Pretrained Convolutions Better than Pretrained Transformers?

In the era of pre-trained language models, Transformers are the de facto choice of model architectures.While recent research has shown promise in entirely convolutional, or CNN, architectures, they have not been explored using the pre-train-fine-tune paradigm.In the context of language models, are convolutional models competitive to Transformers when pre-trained?This paper investigates this research question and presents several interesting findings.Across an extensive set of experiments on 8 datasets/tasks, we find that CNN-based pre-trained models are competitive and outperform their Transformer counterpart in certain scenarios, albeit with caveats.Overall, the findings outlined in this paper suggest that conflating pre-training and architectural advances is misguided and that both advances should be considered independently.We believe our research paves the way for a healthy amount of optimism in alternative architectures.
Towards Universal Physical Attacks on Single Object Tracking

Recent studies show that small perturbations in video frames could misguide single object trackers. However, such attacks have been mainly designed for digital-domain videos (i.e., perturbation on full images), which makes them practically infeasible to evaluate the adversarial vulnerability of trackers in real-world scenarios. Here we made the first step towards physically feasible adversarial attacks against visual tracking in real scenes with a universal patch to camouflage single object trackers. Fundamentally different from physical object detection, the essence of single object tracking lies in the feature matching between the search image and templates, and we therefore specially design the maximum textural discrepancy (MTD), a resolution-invariant and target location-independent feature de-matching loss. The MTD distills global textural information of the template and search images at hierarchical feature scales prior to performing feature attacks. Moreover, we evaluate two shape attacks, the regression dilation and shrinking, to generate stronger and more controllable attacks. Further, we employ a set of transformations to simulate diverse visual tracking scenes in the wild. Experimental results show the effectiveness of the physically feasible attacks on SiamMask and SiamRPN++ visual trackers both in digital and physical scenes.
GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning

Automatic math problem solving has recently attracted increasing attention as a longstanding AI benchmark.In this paper, we focus on solving geometric problems, which requires a comprehensive understanding of textual descriptions, visual diagrams, and theorem knowledge.However, the existing methods were highly dependent on handcraft rules and were merely evaluated on small-scale datasets.Therefore, we propose a Geometric Question Answering dataset GeoQA, containing 5,010 geometric problems with corresponding annotated programs, which illustrate the solving process of the given problems.Compared with another publicly available dataset GeoS, GeoQA is 25 times larger, in which the program annotations can provide a practical testbed for future research on explicit and explainable numerical reasoning.Moreover, we introduce a Neural Geometric Solver (NGS) to address geometric problems by comprehensively parsing multimodal information and generating interpretable programs.We further add multiple self-supervised auxiliary tasks on NGS to enhance cross-modal semantic representation.Extensive experiments on GeoQA validate the effectiveness of our proposed NGS and auxiliary tasks.However, the results are still significantly lower than human performance, which leaves large room for future research.Our benchmark and code are released at https://github.com/chenjudge/GeoQA.
Geometric Variational Inference

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.
Secure boot, trusted boot and remote attestation for ARM TrustZone-based IoT Nodes

With the extensive application of IoT techniques, IoT devices have become ubiquitous in daily lives. Meanwhile, attacks against IoT devices have emerged to compromise IoT devices by tampering with system pre-installed programs or injecting new malware. To mitigate these attacks, integrity enforcement of IoT systems has been proposed. The integrity of an IoT device system includes load-time integrity and runtime integrity. In this paper, we design an IoT system based on ARM TrustZone to enforce the system integrity. First, we establish the root of trust and propose a hybrid booting approach consisting of both secure boot and trusted boot to enforce the system load-time integrity. Second, we investigate a paging-based process integrity measurement method to measure the NW processes and conduct remote attestation based on the measurement results ensuring the NW runtime process integrity. We implement an IoT prototype system on a NXP i.MX6Q SABRE SD development board to assess its feasibility. Real-world experiment results demonstrate that our prototype introduces negligible performance overhead to the original system.
A Color Image Encryption Scheme Based on a Hybrid Cascaded Chaotic System

As image is an important way of information representation, researchers pay more and more attention on image encryption. In order to improve the performance of image encryption, a novel image encryption scheme based on a hybrid cascaded chaotic system and sectoral segmentation is proposed in this paper. Hybrid cascaded chaotic system has a larger key space, higher complexity, more sensitivity to initial conditions. Four chaotic sequences relevant to a plain image are generated by this system, which strengthen plaintext correlation and the randomness. During the scrambling process, sectoral segmentation focuses on how to extract a sequence from the disk storing data, which can not only reduce the correlation between the three components of the image, but also hide image information to a large extent. Further, a DNA algorithm is used in the diffusion process. Simulation shows that the proposed scheme can effectively resist various attacks and improve the encryption performance.
A 28nm 276.55TFLOPS/W Sparse Deep-Neural-Network Training Processor with Implicit Redundancy Speculation and Batch Normalization Reformulation

A dynamic weight pruning (DWP) explored processor, named Trainer, is proposed for energy-efficient deep-neural-network (DNN) training on edge-device. It has three key features: 1) A implicit redundancy speculation unit (IRSU) improves 1.46× throughput. 2) A dataflow, allowing a reuse-adaptive dynamic compression and PE regrouping, increases 1.52× utilization. 3) A data-retrieval eliminated batch-normalization (BN) unit (REBU) saves 37.1% of energy. Trainer achieves a peak energy efficiency of 276.55TFLOPS/W. It reduces 2.23× training energy and offers a 1.76× training speedup compared with the state-of-the-art sparse DNN training processor.
Pose Estimated Yoga Monitoring System

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Training health workers at scale in Nigeria to fight COVID-19 using the InStrat COVID-19 tutorial app: an e-health interventional study

Health worker training is an essential component of epidemic control; rapid delivery of such training is possible in low-middle income countries with digital platforms.Based on prior experience with the Ebola outbreak, we developed and deployed a bespoke InStrat COVID-19 tutorial app, to deliver accurate and regularly updated information about COVID-19 to frontline health workers and epidemic response officers across 25 states of Nigeria. The potential effectiveness of this app in training frontline health workers was assessed through online pre- and post-tests and a survey.A total of 1051 health workers from 25 states across Nigeria undertook the e-learning on the InStrat COVID-19 training app. Of these, 627 (57%) completed both the pre- and post-tests in addition to completing the training modules. Overall, there were statistically significant differences between pre- and post-tests knowledge scores (54 increasing to 74). There were also differences in the subcategories of sex, region and cadre. There were higher post-test scores in males compared with females, younger versus older and southern compared with northern Nigeria. A total of 65 (50%) of the participants reported that the app increased their understanding of COVID-19, while 69 (53%) stated that they had applied the knowledge and skills learnt at work. Overall, the functionality and usability of the app were satisfactory.Capacity building for epidemic control using e-health applications is potentially effective, can be delivered at minimal cost and service disruption and can serve as a tool for capacity building in similar contexts.
Adversarially Learning Disentangled Speech Representations for Robust Multi-Factor Voice Conversion

Factorizing speech as disentangled speech representations is vital to achieve highly controllable style transfer in voice conversion (VC). Conventional speech representation learning methods in VC only factorize speech as speaker and content, lacking controllability on other prosody-related factors. State-of-the-art speech representation learning methods for more speechfactors are using primary disentangle algorithms such as random resampling and ad-hoc bottleneck layer size adjustment,which however is hard to ensure robust speech representationdisentanglement. To increase the robustness of highly controllable style transfer on multiple factors in VC, we propose a disentangled speech representation learning framework based on adversarial learning. Four speech representations characterizing content, timbre, rhythm and pitch are extracted, and further disentangled by an adversarial Mask-And-Predict (MAP)network inspired by BERT. The adversarial network is used tominimize the correlations between the speech representations,by randomly masking and predicting one of the representationsfrom the others. Experimental results show that the proposedframework significantly improves the robustness of VC on multiple factors by increasing the speech quality MOS from 2.79 to3.30 and decreasing the MCD from 3.89 to 3.58.
On the Limit of English Conversational Speech Recognition

In our previous work we demonstrated that a single headed attention encoder-decoder model is able to reach state-of-the-art results in conversational speech recognition. In this paper, we further improve the results for both Switchboard 300 and 2000. Through use of an improved optimizer, speaker vector embeddings, and alternative speech representations we reduce the recognition errors of our LSTM system on Switchboard-300 by 4% relative. Compensation of the decoder model with the probability ratio approach allows more efficient integration of an external language model, and we report 5.9% and 11.5% WER on the SWB and CHM parts of Hub5'00 with very simple LSTM models. Our study also considers the recently proposed conformer, and more advanced self-attention based language models. Overall, the conformer shows similar performance to the LSTM; nevertheless, their combination and decoding with an improved LM reaches a new record on Switchboard-300, 5.0% and 10.0% WER on SWB and CHM. Our findings are also confirmed on Switchboard-2000, and a new state of the art is reported, practically reaching the limit of the benchmark.
MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction

Keyphrase extraction (KPE) automatically extracts phrases in a document that provide a concise summary of the core content, which benefits downstream information retrieval and NLP tasks.Previous state-of-the-art (SOTA) methods select candidate keyphrases based on the similarity between learned representations of the candidates and the document.They suffer performance degradation on long documents due to discrepancy between sequence lengths which causes mismatch between representations of keyphrase candidates and the document.In this work, we propose a novel unsupervised embedding-based KPE approach, Masked Document Embedding Rank (MDERank), to address this problem by leveraging a mask strategy and ranking candidates by the similarity between embeddings of the source document and the masked document.We further develop a KPE-oriented BERT (KPEBERT) model by proposing a novel self-supervised contrastive learning method, which is more compatible to MDERank than vanilla BERT.Comprehensive evaluations on six KPE benchmarks demonstrate that the proposed MDERank outperforms state-of-the-art unsupervised KPE approach by average 1.80 F 1@15 improvement.MDERank further benefits from KPEBERT and overall achieves average 3.53 F 1@15 improvement over the SOTA SIFRank.Our code is available at https: //github.com/LinhanZ/mderank.
Real-Time Twitter Trend Analysis Using Big Data Analytics and Machine Learning Techniques

Twitter is a popular microblogging social media, using which its users can share useful information. Keeping a track of user postings and common hashtags allows us to understand what is happening around the world and what are people’s opinions on it. As such, a Twitter trend analysis analyzes Twitter data and hashtags to determine what topics are being talked about the most on Twitter. Feature extraction and trend detection can be performed using machine learning algorithms. Big data tools and techniques are needed to extract relevant information from continuous steam of data originating from Twitter. The objectives of this research work are to analyze the relative popularity of different hashtags and which field has the maximum share of voice. Along with this, the common interests of the community can also be determined. Twitter trends plan an important role in the business field, marketing, politics, sports, and entertainment activities. The proposed work implemented the Twitter trend analysis using latent Dirichlet allocation, cosine similarity, K means clustering, and Jaccard similarity techniques and compared the results with Big Data Apache SPARK tool implementation. The LDA technique for trend analysis resulted in an accuracy of 74% and Jaccard with an accuracy of 83% for static data. The results proved that the real-time tweets are analyzed comparatively faster in the Big Data Apache SPARK tool than in the normal execution environment.
Learning to extract geographic information from internet router hostnames

Geolocating Internet routers is a long-standing and notoriously difficult challenge, and current solutions lack the accuracy and adaptability to yield reliable results. We revisit this problem, designing a solution capable of accurately and comprehensively extracting geographic information that network operators embed into router interface hostnames. We train our system using dictionaries that map geographic codes to known locations, and constrain inferences with delay measurements conducted from a distributed set of vantage points. While most operators use known geographic codes, some devise their own mnemonic codes for locations, which our system also extracts and interprets.
Design and Synchronization of Chaos-Based True Random Number Generators and Its FPGA Implementation

In this paper, a new chaos-based true random number generator (TRNG) and a sliding mode controller (SMC) to synchronize the proposed TRNGs are proposed.Firstly, the continuous chaotic system is transformed into a discrete system that preserves the original continuous system's chaotic behavior and makes it easy to realize with the field-programmable gate array (FPGA) for synchronization control.Then, a discrete SMC is introduced to solve the synchronization problem of the master-slave discrete chaotic systems.Subsequently, a novel hybrid function integrated with the El-Gamal algorithm is proposed to complete the TRNG design.Finally, the FPGA-based realization of the synchronized master-slave TRNGs is implemented.The randomness quality of the proposed TRNGs has been ensured by using Shannon's entropy and histogram analysis.Furthermore, the National Institute of Standards and Technology (NIST) test suite has also been introduced to evaluate the proposed TRNGs and compare with the existing results in the literature.
An Imbalanced Multifault Diagnosis Method Based on Bias Weights AdaBoost

Fault diagnosis plays an important role in ensuring process safety. It is noted that imbalance between fault data and normal data always exists, and multifault obviously outranges a single fault in common, which leads to more challenges to fault diagnosis. In this article, an imbalanced multifault diagnosis method based on bias weights AdaBoost (BW-AdaBoost) is proposed. First, majority normal samples are under-sampled by K-nearest neighbor (KNN) to collect the boundary samples between majority normal samples and minority fault samples, and then the bias datasets are formed by under-sampled majority samples and minority samples. Second, different weak classifiers with adaptive weights are constructed based on the bias datasets and are integrated into a strong classifier, which is taken as the base classifier. While constructing the weak classifier, higher weights are given to the items corresponding to the minority class in the loss function to enhance the influence of minority class samples. Third, to solve the multifault problem, the base classifiers are integrated into a multiclassification model by the hierarchical structure which needs fewer classifiers and less computational expense. Finally, through simulation experiment, the comparison results show that the proposed imbalanced multifault diagnosis method based on BW-AdaBoost can effectively improve the diagnosis accuracy and F1 score.