Reviewerly/exHarmony huggingface dataset:
On the Role of Text Preprocessing in BERT Embedding-based DNNs for Classifying Informal Texts

Due to highly unstructured and noisy data, analyzing society reports in written texts is very challenging.Classifying informal text data is still considered a difficult task in natural language processing since the texts could contain abbreviated words, repeating characters, typos, slang, et cetera.Therefore, text preprocessing is commonly performed to remove the noises and make the texts more structured.However, we argued that most tasks of preprocessing are no longer required if suitable word embeddings approach and deep neural network (DNN) architecture are correctly chosen.This study investigated the effects of text preprocessing in fine-tuning a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model using various DNN architectures such as multilayer perceptron (MLP), long short-term memory (LSTM), bidirectional long-short term memory (Bi-LSTM), convolutional neural network (CNN), and gated recurrent unit (GRU).Various experiments were conducted using numerous learning rates and batch sizes.As a result, text preprocessing had insignificant effects on most models such as LSTM, Bi-LSTM, and CNN.Moreover, the combination of BERT embeddings and CNN produced the best classification performance.
SciDeBERTa: Learning DeBERTa for Science Technology Documents and Fine-Tuning Information Extraction Tasks

Deep learning-based language models (LMs) have transcended the gold standard (human baseline) of SQuAD 1.1 and GLUE benchmarks in April and July 2019, respectively. As of 2022, the top five LMs on the SuperGLUE benchmark leaderboard have exceeded the gold standard. Even people with good general knowledge will struggle to solve problems in specialized fields such as medicine and artificial intelligence. Just as humans learn specialized knowledge through bachelor&#x2019;s, master&#x2019;s, and doctoral courses, LMs also require a process to develop the ability to understand domain-specific knowledge. Thus, this study proposes SciDeBERTa and SciDeBERTa (CS) as pretrained LMs (PLMs) specialized in the science and technology domain. We further pretrained DeBERTa, which was trained with a general corpus, with the science and technology domain corpus. Experiments verified that SciDeBERTa (CS) continually pretrained in the computer science domain achieved 3.53% and 2.17% higher accuracies than SciBERT and S2ORC-SciBERT, respectively, which are science and technology domain specialized PLMs, in the task of recognizing entity names in the SciERC dataset. In the JRE task of the SciERC dataset, SciDeBERTa (CS) achieved a 6.7% higher performance than the baseline SCIIE. In the GENIA dataset, SciDeBERTa achieved the best performance compared to S2ORC-SciBERT, SciBERT, BERT, DeBERTa and SciDeBERTa (CS). Furthermore, re-initialization technology and optimizers after Adam were explored during fine-tuning to verify the language understanding of PLMs.
MNIST Image Classification Using Convolutional Neural Networks

Convolutional neural network (CNN) holds the current research interest in the ever-evolving image classification field. Accurate classifying the image data with minimum of time is highly desired. But the traditional CNN architecture often fails to generate the appropriate outcome for large dataset. So, a modified approach of CNN is proposed here which is the combination of data augmentation and batch normalization embedded with CNN. Nowadays, identifying or classifying digits accurately with variety of modes is really a task of challenge. The advantages of the proposed approach are noted when it is applied to the popular MNIST dataset used for digit classification. The proposed approach has been compared with some existing techniques and results infer that the validation, training loss, and testing accuracy of the proposed approach are more superior as compared to the state-of-art approaches.
Focus on the Target’s Vocabulary: Masked Label Smoothing for Machine Translation

Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models.However, we argue that simply applying both techniques can be conflicting and even leads to sub-optimal performance.When allocating smoothed probability, original label smoothing treats the source-side words that would never appear in the target language equally to the real target-side words, which could bias the translation model.To address this issue, we propose Masked Label Smoothing (MLS), a new mechanism that masks the soft label probability of source-side words to zero.Simple yet effective, MLS manages to better integrate label smoothing with vocabulary sharing.Our extensive experiments show that MLS consistently yields improvement over original label smoothing on different datasets, including bilingual and multilingual translation from both translation quality and model's calibration.Our code is released at PKUnlp-icler.
Cross-Modal Cloze Task: A New Task to Brain-to-Word Decoding

Decoding language from non-invasive brain activity has attracted increasing attention from both researchers in neuroscience and natural language processing.Due to the noisy nature of brain recordings, existing work has simplified brain-to-word decoding as a binary classification task which is to discriminate a brain signal between its corresponding word and a wrong one.This pairwise classification task, however, cannot promote the development of practical neural decoders for two reasons.First, it has to enumerate all pairwise combinations in the test set, so it is inefficient to predict a word in a large vocabulary.Second, a perfect pairwise decoder cannot guarantee the performance on direct classification.To overcome these and go a step further to a realistic neural decoder, we propose a novel Cross-Modal Cloze (CMC) task which is to predict the target word encoded in the neural image with a context as prompt.Furthermore, to address this task, we propose a general approach that leverages the pre-trained language model to predict the target word.To validate our method, we perform experiments on more than 20 participants from two brain imaging datasets.Our method achieves 28.91% top-1 accuracy and 54.19% top-5 accuracy on average across all participants, significantly outperforming several baselines.This result indicates that our model can serve as a state-of-the-art baseline for the CMC task.More importantly, it demonstrates that it is feasible to decode a certain word within a large vocabulary from its neural brain activity.
Let the Big Data Speak: Collaborative Model of Topic Extract and Sentiment Analysis COVID-19 Based on Weibo Data

Micro-blog is an important medium of emergency communication. The topic and emotion analysis of micro-blog is of great significance in identifying and predicting potential problems and risks. In this paper, a collaborative analysis model of emotion and topic mining is constructed to analyze the users’ sentiment and the topics they care about, Firstly, we use SO-PMI to construct domain sentiment lexicon and extract topics with LDA. Then we use the collaborative model to analyze sentiment and topic. The results showed that the model we proposed can present the features of sentiment and topic of user concerns. And through text clustering and sentiment analysis, it is found that the attitude of users towards the COVID-19 has gone through three stages, namely, a period of fluctuating tension and anxiety, a period of slowly rising solidarity and a period of stable self-confidence with little fluctuation, on the whole, positive is greater than negative, positive than negative state.
Language-agnostic Semantic Consistent Text-to-Image Generation

Recent GAN-based text-to-image generation models have advanced that they can generate photo-realistic images matching semantically with descriptions. However, research on multi-lingual text-to-image generation has not been carried out yet much. There are two problems when constructing a multilingual text-to-image generation model: 1) language imbalance issue in text-to-image paired datasets and 2) generating images that have the same meaning but are semantically inconsistent with each other in texts expressed in different languages. To this end, we propose a Language-agnostic Semantic Consistent Generative Adversarial Network (LaSC-GAN) for text-to-image generation, which can generate semantically consistent images via language-agnostic text encoder and Siamese mechanism. Experiments on relatively low-resource language text-image datasets show that the model has comparable generation quality as images generated by high-resource language text, and generates semantically consistent images for texts with the same meaning even in different languages.
Offline Handwritten Mathematical Expression Recognition via Graph Reasoning Network

Handwritten mathematical expression recognition (HMER) remains a challenge due to the complex 2-D structure layout and variable writing styles. Recently, large progress has been made by using deep encoder-decoder networks, which treat HMER as an Image-to-Sequence task and parse the math expression into a sequence (i.e. LaTeX). However, (1) mathematical expression is a 2-D structure pattern and sequence representation can not explicitly explore the structural relationship between symbols. (2) Image-to-Sequence as recurrent models can not infer in parallel during test stage. In this paper, we formulate mathematical expression recognition as an Image-to-Graph task and propose a Graph Reasoning Network (GRN) for offline HMER task. Compared with sequence representation, graph representation is more interpretable and more consistent with human visual cognition. Our method builds graph on math symbols detected from image, aggregates node and edge features via a Graph Neural Network (GNN) and parses the graph to give Symbol Layout Tree (SLT) format recognition result via node and edge classification. Experiments on public datasets show that our model achieve competitive results against other methods and can interpret the located symbols and inter-relationship explicitly.
Linguistically Enhanced Text to Sign Gloss Machine Translation

In spite of the recent advances in Machine Translation (MT) for spoken languages, translation between spoken and Sign Languages (SLs) or between Sign Languages remains a difficult problem. Here, we study how Neural Machine Translation (NMT) might overcome the communication barriers for the Deaf and Hard-of-Hearing (DHH) community. Namely, we approach the Text2Gloss translation task in which spoken text segments are translated to lexical sign representations. In this context, we leverage transformer-based models via (1) injecting linguistic features that can guide the learning process towards better translations; and (2) applying a Transfer Learning strategy to reuse the knowledge of a pre-trained model. To this aim, different aggregation strategies are compared and evaluated under Transfer Learning and random weight initialization conditions. The results of this research reveal that linguistic features can successfully contribute to achieve more accurate models; meanwhile, the Transfer Learning procedure applied conducted to substantial performance increases.
Cryptanalysis and Improvement of DeepPAR: Privacy-Preserving and Asynchronous Deep Learning for Industrial IoT

Industrial Internet of Things (IIoT) is gradually changing the mode of traditional industries with the rapid development of big data. Besides, thanks to the development of deep learning, it can be used to extract useful knowledge from the large amount of data in the IIoT to help improve production and service quality. However, the lack of large-scale data sets will lead to low performance and overfitting of learning models. Therefore, federated deep learning with distributed data sets has been proposed. Nevertheless, the research has shown that federated learning can also leak the private data of participants. In IIoT, once the privacy of participants in some special application scenarios is leaked, it will directly affect national security and people’s lives, such as smart power grid and smart medical care. At present, several privacy-preserving federated learning schemes have been proposed to preserve data privacy of participants, but security issues prevent them from being fully applied. In this article, we analyze the security of the DeepPAR scheme proposed by Zhang et al., and point out that the scheme is insecure in the re-encryption key generation process, which will cause the leakage of the secret key of participants or the proxy server. In addition, the scheme is not resistant to collusion attacks between the parameter server and participants. Based on this, we propose an improved scheme. The security proof shows that the improved scheme solves the security problem of the original scheme and is resistant to collusion attacks. Finally, the security and accuracy of the scheme is illustrated by performance analysis.
A Transfer Learning Approach for Face Recognition Using Average Pooling and MobileNetV2

Facial recognition is a fundamental method in facial-related science such as face detection, authentication, monitoring, and a crucial phase in computer vision and pattern recognition. Face recognition technology aids in crime prevention by storing the captured image in a database, which can then be used in various ways, including identifying a person. With just a few faces in the frame, most facial recognition systems function sufficiently when the techniques have been tested under artificial illumination, with accurate facial poses and non-blurry images. In our proposed system, a face recognition system is proposed using average pooling and MobileNetV2. The classifiers are implemented after a set of preprocessing steps on the retrieved image data. To compare the model is more effective, a performance test on the result is performed. It is observed from the study that MobileNetV2 triumphs over average pooling with an accuracy rate of 98.89% and 99.01% on training and test data, respectively.
Modern approaches to career guidance of school students

We consider the current formats of career guidance work, which, firstly, are focused on solving the problem of attracting applicants to universities, secondly, take into account the interests of students and contribute to conscious professional self-determination, and thirdly, take into account the labor market situation. The solution of these problems is possible as a result of the systematic organization of career guidance, the key subject of which is the university, while ensuring interaction in the triad “school – university – employer”. Within the framework of the university, career guidance work can be of a tiered nature and be carried out at the level of an educational program, department, institute/faculty, university. Domestic and foreign experience indicates the need to increase the depth of career guidance work, which is facilitated by the format of the “kids university”, in which students of primary and secondary schools get acquainted with professional areas. One of the urgent problems of career guidance work with school students is to increase their involvement in the proposed activities. The involvement and, as a result, the effectiveness of career guidance work makes it possible to increase its practical nature (trainings, master classes, professional tests), the use of a competitive component (olympiads, grant competitions), the use of game components (gamification elements) in career guidance events.
AI Film Creation Oriented Transformation in the Era of Artificial Intelligence

AI film is a commercial film genre with distinctive technical nature and profound thought, which includes two branches: AI film on the level of technical practicality and AI film on the level of subject creation.With the rapid development of science and technology, creation orientations of AI films have also changed.The reason is that the art world is a reflection of the real world, and is also viewed and reconstructed by the real world.Perceptions and concepts of the real world will have an indirect impact on the foundation of the art world, and artistic ideas can in turn become the trigger and starting point for real world developments.Nowadays, against the backdrop of deep impacts brought by artificial intelligence in everyday life, clarifying AI film creation-oriented transformation and its underlying logic shed light on current artistic creations of films and TV productions.
Implementation and Comparison of Big Data Analysis on Large Dataset Using SPARK and FLINK

Various big data challenges are addressed in this paper. We encounter many difficulties in the storage and processing of huge data sets. Various open source and enterprise tools are available for the processing of big data. The choice for the best tool to be used for analysis is very important to make. The tool selected must be able to provide the accurate analysis results and in real time. The advantages and disadvantages of the various tools are also discussed. In the proposed work, a comparison between the two Big Data processing frameworks is made. Big Data deals with two types of processing data i.e. batch data, in which blocks of data are executed with no time constraint, and stream data, in which continuous data input is processed in near real time. In the proposed work, Spark and Flink are used as the processing systems. A parking dataset containing a large number of records is created, which is then processed with these two processing frameworks. Spark due to its batch processing nature shows some lag in the processing of the data, whereas Flink on the other hand, with its stream processing nature certainly has an edge over Spark. To understand the results in a better way, a graphical representation depicting the execution time to process a number of records is also made. This graphical representation clearly helps in visualizing the comparison between the two processing frameworks, due to which a quick judgment can be easily made about which system is better than the other.
Organizational Citizenship Behavior and Job Satisfaction from The Nurses' Perspective

research in Saudi Arabia has been published investigating organizational citizenship behavior among nurses. In addition, job satisfaction has been identified as the main solution to the high turnover rate among nurses in Saudi Arabia and one of the factors that could affect organizational citizenship behavior.&#x0D; Aim: The current study aimed to determine the level of organizational citizenship behavior, assess the level of job satisfaction, and identify the relationship between organizational citizenship behaviors and job satisfaction from the nurses’ perspective.&#x0D; Methods: A quantitative cross-sectional correlational study was conducted at the two largest government hospitals in Medina City, Saudi Arabia. A convenience sampling technique was used to recruit 315 nurses. The study used an electronic self-reporting questionnaire consisting of sociodemographic characteristics, an organizational citizenship behavior scale, and a nurse job satisfaction scale.&#x0D; Results: In this study, the overall level of organizational citizenship behavior among nurses was high (3.86±0.35). The overall level of nurses' job satisfaction was medium (2.88±0.76). There was a statistically significant positive relationship between overall organizational citizenship behavior and overall job satisfaction among nurses (r = 0.354, p-value &lt;0.01).&#x0D; Conclusion: This study provides evidence that increasing the level of job satisfaction among nurses will increase their levels of organizational citizenship behavior. Therefore, healthcare organizations must focus on certain interventions that could increase nurses' jobs satisfaction, such as providing adequate remuneration, increasing the capacity of nursing schools, emphasizing psychological support and participative leadership, and improving the community’s perception of the nursing profession.
A multipurpose mobile manipulator for autonomous firefighting and construction of outdoor structures

This paper describes the implementation of a multipurpose, autonomous, mobile manipulator for building outdoor structures and for firefighting. Such a system finds applications in different industrial automation, manufacturing, and civil-construction scenarios as well as in search-and-rescue operations. This system was developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020, showcasing once again the role of major scientific competitions in advancing the state of the art and exploring solutions to open problems. The paper presents in detail the hardware and software architectures of the developed mobile manipulator, while proposing methods for: a) Building outdoor structures consisting of heterogeneous brick patterns, and b) Entering buildings to locate and extinguish fires. Solutions were successfully deployed in the near-realistic arenas of the MBZIRC 2020 competition and resulted in the first-place award for the firefighting scenario.
A Syntax-enhanced model based on category keywords for biomedical relation extraction

Certain categories in multi-category biomedical relationship extraction have linguistic similarities to some extent. Keywords related to categories and syntax structures of samples between these categories have some notable features, which are very useful in biomedical relation extraction. The pre-trained model has been widely used and has achieved great success in biomedical relationship extraction, but it is still incapable of mining this kind of information accurately. To solve the problem, we present a syntax-enhanced model based on category keywords. First, we prune syntactic dependency trees in terms of category keywords obtained by the chi-square test. It reduces noisy information caused by current syntactic parsing tools and retains useful information related to categories. Next, to encode category-related syntactic dependency trees, a syntactic transformer is presented, which enhances the ability of the pre-trained model to capture syntax structures and to distinguish multiple categories. We evaluate our method on three biomedical datasets. Compared with state-of-the-art models, our method performs better on these datasets. We conduct further analysis to verify the effectiveness of our method.
IDEA: Increasing Text Diversity via Online Multi-Label Recognition for Vision-Language Pre-training

Vision-Language Pre-training (VLP) with large-scale image-text pairs has demonstrated superior performance in various fields. However, the image-text pairs co-occurrent on the Internet typically lack explicit alignment information, which is suboptimal for VLP. Existing methods proposed to adopt an off-the-shelf object detector to utilize additional image tag information. However, the object detector is time-consuming and can only identify the pre-defined object categories, limiting the model capacity. Inspired by the observation that the texts incorporate incomplete fine-grained image information, we introduce IDEA, which stands for increasing text diversity via online multi-label recognition for VLP. IDEA shows that multi-label learning with image tags extracted from the texts can be jointly optimized during VLP. Moreover, IDEA can identify valuable image tags online to provide more explicit textual supervision. Comprehensive experiments demonstrate that IDEA can significantly boost the performance on multiple downstream datasets with a small extra computational cost.