Reviewerly/exHarmony huggingface dataset:
Chinese Drug Information Named Entity Recognition Based on MSCNN And BERT-BiLSTM-CRF

In the study of drug-drug interactions, knowledge graph on drugs plays an important role to extract drug features. Named entity recognition is a key to construct knowledge graph. The traditional method BERT-BiLSTM-CRF is usually effective in extracting entities from the corpus. However, in the field of Chinese drug information, the statement format of the corpus is not uniform, and the entities are long and semantically complex, such as in the extraction of key entities such as adverse reactions, indications and contraindications in drug instructions, the traditional methods cannot effectively deal with the above difficulties. In this work, we propose a CDINER model to solve the Chinese drug information named entity recognition. The model performs a multi-scale convolution after the character vector are generated by the BERT model, and then to be input to the Bi-LSTM. Then the input and output of the BiLSTM module are concatenated by skip connect and fed to the CRF module. In the experiments, we compare our proposed method with other classical models in the Chinese drug information corpus. The results show that our model outperforms others. It can effectively identify three types of drug information entities that consist of adverse reactions, indications and contraindications.
Global-aware Pyramid Network with Boundary Adjustment for Anchor-free Temporal Action Detection

Fine-grained temporal action detection aims at predicting the categories and locating the boundaries of fine-grained action instances in long, untrimmed videos. The fine-grained classification of action instances brings new challenges to temporal action detection, which changes the distribution of action instances of different durations and increases the proportion of short action instances. However, the existing anchor-free detection methods cannot fully utilize global information and local information. Therefore, this paper proposes an anchor-free temporal action detection method with global feature enhancement and local boundary adjustment. Based on the feature pyramid, the attention mechanism of transformer is used to model long-range temporal dependencies between features at different locations of the same level and introduce global information from the upper-level of the feature pyramid to generate coarse predictions. To obtain local details, the interaction with the low-level feature is used to adjust the boundaries of coarse predictions. Experiments on FineAction demonstrate the effectiveness of this method.
Incorporating Typological Information into Pretrained Word Representations for Improved Performance

Typological information of a language such as head finality, consonant clusters, etc. has been used to enhance the accuracy of multilingual dependency parsers. In this paper, we study if pretrained word vectors enhanced with typological information capture linguistic information more accurately leading to improved performance across several linguistic and downstream tasks. We present an approach to incorporate typological information into pretrained word representations using attention and evaluate the effectiveness of the approach by probing enhanced representations, and measuring accuracy and MDL codelengths. Our study shows that for dependency relations and parts-of-speech in English, the enhanced pretrained word representations have shorter codelengths exhibiting stronger regularity than the original pretrained word representations while achieving comparable probing accuracy. We also show that the English enhanced pretrained word representations can better encode certain known linguistic features such as entailment. Finally, we illustrate the potential benefits of English enhanced pretrained word representations by obtaining a 3 point ROUGE-1 score and 2 point ROUGE-L score increase for a text summarization task.
The mr-MDA: An Invariant to Shifting, Scaling, and Rotating Variance for 3D Object Recognition Using Diffractive Deep Neural Network

The diffractive deep neural network (D2NN) can efficiently accomplish 2D object recognition based on rapid optical manipulation. Moreover, the multiple-view D2NN array (MDA) possesses the obvious advantage of being able to effectively achieve 3D object classification. At present, 3D target recognition should be performed in a high-speed and dynamic way. It should be invariant to the typical shifting, scaling, and rotating variance of targets in relatively complicated circumstances, which remains a shortcoming of optical neural network architectures. In order to efficiently recognize 3D targets based on the developed D2NN, a more robust MDA (mr-MDA) is proposed in this paper. Through utilizing a new training strategy to tackle several random disturbances introduced into the optical neural network system, a trained mr-MDA model constructed by us was numerically verified, demonstrating that the training strategy is able to dynamically recognize 3D objects in a relatively stable way.
PointIt3D: a benchmark dataset and baseline for pointed object detection task

Pointed object detection is of great importance for human-machine interaction, but attempts to solve this task may run into the difficulties of lack of available large scale datasets since people hardly record 3D scenes with a human pointing at specific objects. In efforts to mitigate this gap, we cultivate the first benchmark dataset for this task: PointIt3D (available at https://pan.baidu.com/share/init?surl=E3u96E7dEXnrR1dDris_1w (access code: jps5)), containing 347 scans now and can be easily scaled up to facilitate future utilizations, which is automatically constructed from existing 3D scenes from ScanNet1 and 3D people models using our novel synthetic algorithm that achieves a high acceptable rate of more than 85% according to three experts’ assessments, which hopefully would pave the way for further studies. We also provide a simple yet effective baseline based on anomaly detection and majority voting pointline generation to solve this task based on our dataset, which achieves accuracy of 55.33%, leaving much room for further improvements. Code will be released at https://github.com/XHRlyb/PointIt3D.
Self-supervision based super-resolution approach for light field refocused image

Light field imaging can record spatial and angular information of scenes simultaneously, which can provide images focused at different depths by computational imaging. However, the number of sensor pixels and the size of the microlens array limit the resolution of refocused images, which makes them difficult to be used for downstream tasks. To overcome this limitation, we propose a self-supervised super-resolution algorithm to increase the resolution of refocused images, which relies only on the image prior information. With the prior information of low-resolution refocused images and convolutional structure, we can not only significantly improve image quality, but also solve the problem of insufficient training data. Intensive experiments show that the proposed self-supervised approach is able to obtain impressive results and is even comparable to the data-hungry supervised learning methods.
Concepts encoding via knowledge-guided self-attention networks

With the growth of digital data created by us, a large number of deep learning models have been proposed for data mining. Representation learning offers an exciting avenue to address data mining demands by embedding data into feature space. In the healthcare field, most existing methods are proposed to mine electronic health records (EHR) data by learning medical concept representations. Despite the vigorous development of this field, we find the contextual information of medical concepts has always been overlooked, which is important to represent these concepts. Given these limitations, we design a novel medical concept representation method, which is equipped with a self-attention mechanism to learn contextual representation from EHR data and prior knowledge. Extensive experiments on medication recommendation tasks verify the designed modules are consistently beneficial to model performance.
Accurate neuroanatomy segmentation using 3D spatial and anatomical attention neural networks

Brain structure segmentation from 3D magnetic resonance (MR) images is a prerequisite for quantifying brain morphology. Since typical 3D whole brain deep learning models demand large GPU memory, 3D image patch-based deep learning methods are favored for their GPU memory efficiency. However, existing 3D image patch-based methods are not well equipped to capture spatial and anatomical contextual information that is necessary for accurate brain structure segmentation. To overcome this limitation, we develop a spatial and anatomical context-aware network to integrate spatial and anatomical contextual information for accurate brain structure segmentation from MR images. Particularly, a spatial attention block is adopted to encode spatial context information of the 3D patches, an anatomical attention block is adopted to aggregate image information across channels of the 3D patches, and finally the spatial and anatomical attention blocks are adaptively fused by an element-wise convolution operation. Moreover, an online patch sampling strategy is utilized to train a deep neural network with all available patches of the training MR images, facilitating accurate segmentation of brain structures. Ablation and comparison results have demonstrated that our method is capable of achieving promising segmentation performance, better than state-of-the-art alternative methods by 3.30% in terms of Dice scores.
Blind image quality assessment based on transformer

Transformer has achieved milestones in natural language processing (NLP). Due to its excellent global and remote semantic information interaction performance, it has gradually been applied in vision tasks. In this paper, we propose PTIQ, which is a pure Transformer structure for Image Quality Assessment. Specifically, we use Swin Transformer Blocks as backbone to extract image features. The extracted feature vectors after extra state embedding and position embedding are fed into the original transformer encoder. Then, the output is passed to the MLP head to predict quality score. Experimental results demonstrate that the proposed architecture achieves outstanding performance.
High-speed videogrammetric technique for displacement measurement of floating offshore wind turbine model

The issue of displacement measurement of floating wind turbine model when experiencing wind-wave loads is one important assessment indicator of stability, but conventional contact methods hardly to utilize in environment which are complex and have limited accessibility. Consequently, we develop the high-speed videogrammetry technique for displacement measurement from tremendous image sequences data, which possesses advantages of non-contact, high frame rate, three-dimensional dense measurement when compare to traditional approaches. The first, the motion of model in image sequences are acquired with the accurate target recognition and tracking algorithms. The second, the cameras interior and exterior parameters are determined through camera calibration and bundle adjustment respectively. The third, the target spatial position and displacement in X, Y, and Z directions are finally calculated based on videogrammetry theory. The experiment results demonstrate that the spatial position calculated by proposed approach can reach submillimeter accuracy in three directions when compare to high accuracy total station. In addition, the credibility of displacement results is further proved by the frequency consistence between measured and theoretical.
No-reference video quality assessment using data dimensionality reduction and attention-based pooling

This paper proposes a new end-to-end no-reference (NR) video quality assessment (VQA) algorithm that makes use of dimensionality reduction and attention-based pooling. Firstly, the dataset is expanded through data enhancement based on frame sampling. Secondly, the cropped video blocks are input into the trainable data dimensionality reduction module which adopts 3D convolution to reduce the dimension of the data. Then, the dimensionality reduced data is input into the backbone of the algorithm to extract spatial features. The extracted features are pooled through attention-based pooling. Finally, the pooled features are regressed to the quality score through the full connection layer. Experimental results show that the proposed algorithm has achieved competitive performance on the LIVE, LIVE Mobile and CVD2014 datasets, and has low complexity.
Attention-guided feature fusion network for crowd counting

How to solve the scale variation and background interference faced by crowd counting algorithms in practical applications is still an open problem. In this paper, to tackle the above problems, we propose the Attention-guided Feature Fusion Network (AFFNet) to learn the mapping between the crowd image and density map. In this network, the Channel-attentive Receptive Field Block (CRFB) is constructed by parallel convolutional layers with different expansion rates to extract multi-scale features. By adopting attention masks generated by high-level features to adjust low-level features, the Feature Fusion Module (FFM) can alleviate the background interference problem at the feature level. In addition, the Double Branch Module (DBM) generates a density estimation map, which further erases the background interference problem at the density level. Extensive experiments conducted on several challenging benchmark datasets including ShanghaiTech, UCF-QNRF and JHU-CROWD++ demonstrate our proposed method is superior to the state-of-the-art approaches.
Mineralization information extraction of metal mining area based on Landsat images

Remote sensing technology can quickly identify hydro thermal alteration related to mineralization and provide help for prospecting through efficient and accurate analysis. In this study, the Zhaoping fault zone is taken as the research area, Landsat8 OLI remote sensing images with good imaging quality are used as the data source, and ENVI software is used to perform pruning, radiometric calibration, atmospheric correction, and preprocessing of removing interference information including water and vegetation from remote sensing images in 2015 and 2020 respectively. The mineralized alteration information of this area is extracted by CROSTA method, and the extracted thematic information is analyzed and verified. In addition, by comparing the extraction maps of mineralization information in 2015 and 2020, it is found that the situation of over exploitation of mineral resources still exists. The results are in good agreement with the known ore points in the study area, indicating that the extraction of metal mineralization information through remote sensing images is of great significance to the study and rational utilization of mineral resources.
No-reference stereoscopic video quality assessment based on Tchebichef moment

We propose a no-reference (NR) stereoscopic video quality assessment (SVQA) model based on Tchebichef moment in this paper. Specifically, we extract keyframes according to mutual information between adjacent frames, and then the extracted keyframes are segmented to patches to calculate low-order Tchebichef moments. Since the strong description ability of Tchebichef moment, and different order of Tchebichef moment can represent independent features with minimal information redundancy, we extract statistical features of Tchebichef moment on computed patches as spatial features. Considering the influence of distortions in spatiotemporal domain to video quality, we use the three-dimensional derivative of Gaussian filters to calculate the spatiotemporal energy responses and extract statistical features from the responses as spatiotemporal features. Finally, we combine the spatial and spatiotemporal features to predict the quality of stereoscopic videos. The proposed model is evaluated on the NAMA3DS1-COSPAD1, SVQA and Waterloo IVC phase I databases. The experimental results show that the proposed model achieved competitive performance as compared with existing SVQA models.
Pre-rotation only at inference-time: a way to rotation invariance

Weight sharing across different locations makes Convolutional Neural Networks (CNNs) space shift invariant, i.e., the weights learned in one location can be applied to recognize objects in other locations. However, weight sharing mechanism has been lacked in Rotated Pattern Recognition (RPR) tasks, and CNNs have to learn training samples in different orientations by rote. As such rote-learning strategy has greatly increased the difficulty of training, a new solution for RPR tasks, Pre-Rotation Only At Inference time (PROAI), is proposed to provide CNNs with rotation invariance. The core idea of PROAI is to share CNN weights across multiple rotated versions of the test sample. At the training time, a CNN was trained with samples only in one angle; at the inference-time, test samples were pre-rotated at different angles and then fed into the CNN to calculate classification confidences; at the end both the category and the orientation were predicted using the position of the max value of these confidences. By adopting PROAI, the recognition ability learned at one orientation can be generalized to patterns at any other orientation, and both the number of parameters and the training time of CNN in RPR tasks can be greatly reduced. Experiments show that PROAI enables CNNs with less parameters and training time to achieve state-of-the-art classification and orientation performance on both rotated MNIST and rotated Fashion MNIST datasets.
SlowFast with DropBlock and smooth samples loss for student action recognition

Due to the advent of large-scale video datasets, action recognition using three-dimensional convolutions (3D CNNs) containing spatiotemporal information has become mainstream. Aiming at the problem of classroom student behavior recognition, the paper adopts the improved SlowFast network structure to deal with spatial structure and temporal events respectively. First, DropBlock (a regularization method) is added to the SlowFast network to solve the overfitting problem. Second, for the problem of Long-Tailed Distribution, the designed Smooth Sample (SS) Loss function is added to the network to smooth the number of samples. Classification experiments show that compared with similar methods, the model accuracy of our method on the Kinetics and Student Action Dataset is increased by 2.1% and 2.9%, respectively.
Real-time image distortion correction based on FPGA

As the primary method for real-time image processing, a field-programmable gate array (FPGA) is widely used in binocular vision systems. Distortion correction is an important component of binocular stereo vision systems. When implementing a real-time image distortion correction algorithm on FPGA, problems, such as insufficient on-chip storage space and high complexity of coordinate correction calculation methods, occur. These problems are analyzed in detail in this study. On the basis of the reverse mapping method, a distortion correction algorithm that uses a lookup table (LUT) is proposed. A compression with restoration method is established for this LUT to reduce space occupation. The corresponding cache method of LUT and the image data are designed. The algorithm is verified on our binocular stereo vision system based on Xilinx Zynq-7020. The experiments show that the proposed algorithm can achieve real-time and high precision gray image distortion correction effect and significantly reduce the consumption of on-chip resources. Enough to meet the requirements of accurate binocular stereo vision system.
LDAM: line descriptors augmented by attention mechanism

Compared with point features, line features can provide more geometric information in vision tasks. Although traditional line descriptor methods have been proposed for a long time, learning-based line descriptor methods still need to be strengthened. Inspired by the message passing mechanism of graph neural networks, we propose a new neural network architecture named LDAM that alternately uses two attention mechanisms to augment line descriptors and extract more line correspondences. Compared with previous methods, our method learns the geometric properties and prior knowledge of images through the mutual aggregation of features between a pair of images. The experiments on real data verify the good performance of LDAM in terms of matching accuracy. Furthermore, LDAM is also robust to viewpoint change or occlusion.
Masked facial region recognition using human pose estimation and broad learning system

COVID-19 and its variants have been posing a large risk to people around the world since the outbreak of the disease. Many techniques like AI are explored to help combat epidemics. People are required or forced to wear a mask to fight against COVID-19 epidemics worldwide. It brings new challenges to the task of masked facial region recognition. When facial regions are occluded by masks, it will result in some failures of face detection algorithms. In this paper, we propose a method to recognize masked faces. It mainly includes three parts. Firstly, the human pose is estimated to produce a series of key points. It is implemented by OpenPose. Secondly, a key-points location strategy is designed to capture the masked facial regions. It can locate the positions of faces accurately. Thirdly, the broad learning system, which is also an incremental learning algorithm, is employed to recognize the classes of candidate regions. Experiments conducted on some datasets shed light on the effectiveness of the proposed method.
Orthogonal matching pursuit with adaptive restriction for sparse subspace clustering

Subspace clustering methods have been used for unsupervised learning and Sparse Subspace Clustering (SSC) is one of the most popular methods. Since ℓ1 optimization in SSC requires complex calculation, Orthogonal Matching Pursuit (OMP) is adopted in OMP-SSC to reduce calculation time, but its performance is unsatisfactory. In this paper, a new algorithm, Orthogonal Matching Pursuit with Adaptive Restriction for Sparse Subspace Clustering (OMPAR-SSC) is proposed, in which two adaptive restrictions varying with the strength or density of connections are developed. Our algorithm can improve the connectivity of the affinity graph and enhance the segmentation effect. Experiments on both synthetic data and real-world data also demonstrate that OMPAR-SSC outperforms other subspace clustering algorithms in terms of accuracy and achieves a good trade-off between efficiency and effectiveness.
RHDDNet: multi-label classification-based detection of image hybrid distortions

Image distortion detection is a key step in image quality assessment and image reconstruction algorithms. In previous work, a large number of research focus on detecting the single distortion in the image. However, the number of distortion types in the image is often uncertain. Thus, we propose a model that can be used for hybrid distortion detection. Concretely, we transform the hybrid distortion detection task into a multi-label classification task and abstract it as a convolutional network optimization problem. A dataset is created to train the model and evaluate its performance. Experiments show that the proposed model performs well in the detection of hybrid distortions in images.