Reviewerly/exHarmony huggingface dataset:
CHOP: An orthogonal hashing method for zero-shot cross-modal retrieval

Cross-modal retrieval has recently attracted much attention because it helps users retrieve data across different modalities. However, with the explosive growth of data, a large number of new emerging concepts (unseen classes) that have not been appeared in the training data (seen classes) bring great challenges to the traditional cross-modal retrieval. Nevertheless, most existing approaches mainly focus on improving cross-modal retrieval performance of seen classes, which may fail in the unseen classes. To address the challenge of zero-shot cross-modal retrieval, we propose an orthogonal method in this paper, i.e., Cross-modal Hashing with Orthogonal Projection (CHOP). It projects cross-modal features and class attributes onto a Hamming space, where each projection of cross-modal features is orthogonal to the mismatched class attributes. By so doing, the model can learn a discriminative and binary representation of each modality. In addition, the class attributes build a bridge to transfer knowledge from seen classes to unseen classes. Furthermore, the orthogonal constraint on binary codes can help to mitigate the hubness problem. Extensive experiments on three benchmark datasets show that the proposed CHOP is effective in handling zero-shot cross-modal retrieval.
Knowledge models from PDF textbooks

Textbooks are educational documents created, structured and formatted by domain experts with the primary purpose to explain the knowledge in the domain to a novice. Authors use their understanding of the domain when structuring and formatting the content of a textbook to facilitate this explanation. As a result, the formatting and structural elements of textbooks carry the elements of domain knowledge implicitly encoded by their authors. Our paper presents an extensible approach towards automated extraction of knowledge models from textbooks and enrichment of their content with additional links (both internal and external). The textbooks themselves essentially become hypertext documents where individual pages are annotated with important concepts in the domain. The evaluation experiments examine several aspects and stages of the approach, including the accuracy of model extraction, the pragmatic quality of extracted models using one of their possible applications— semantic linking of textbooks in the same domain, the accuracy of linking models to external knowledge sources and the effect of integration of multiple textbooks from the same domain. The results indicate high accuracy of model extraction on symbolic, syntactic and structural levels across textbooks and domains, and demonstrate the added value of the extracted models on the semantic level.
Analysis on Effectiveness of Surrogate Data-Based Laser Chaos Decision Maker

The laser chaos decision maker has been demonstrated to enable ultra-high-speed solutions of multiarmed bandit problems or decision-making in the GHz order. However, the underlying mechanisms are not well understood. In this paper, we analyze the chaotic dynamics inherent in experimentally observed laser chaos time series via surrogate data and further accelerate the decision-making performance via parameter optimization. We first evaluate the negative autocorrelation in a chaotic time series and its impact on decision-making detail. Then, we analyze the decision-making ability using three different surrogate chaos time series to examine the underlying mechanism. We clarify that the negative autocorrelation of laser chaos improves decision-making and that the amplitude distribution of the original laser chaos time series is not optimal. Hence, we introduce a new parameter for adjusting the amplitude distribution of the laser chaos to enhance the decision-making performance. This study provides a new insight into exploiting the supremacy of chaotic dynamics in artificially constructed intelligent systems.
The Application of Artificial Intelligence Technology in UAV

From the key technology level, the application of artificial intelligence (AI) in unmanned aerial vehicle (UAV) is discussed. The basic connotation of AI technology is introduced. In the four fields which are machine learning, expert system, automatic planning and distributed artificial intelligence, the application of AI technology in UAV is discussed. The future development trend of UAV based on AI is prospected, which provides a reference for promoting the application of AI in UAV.
FAN: Frequency Aggregation Network for Real Image Super-Resolution

Single image super-resolution (SISR) aims to recover the high-resolution (HR) image from its low-resolution (LR) input image. With the development of deep learning, SISR has achieved great progress. However, It is still a challenge to restore the real-world LR image with complicated authentic degradations. Therefore, we propose FAN, a frequency aggregation network, to address the real-world image super-resolu-tion problem. Specifically, we extract different frequencies of the LR image and pass them to a channel attention-grouped residual dense network (CA-GRDB) individually to output corresponding feature maps. And then aggregating these residual dense feature maps adaptively to recover the HR image with enhanced details and textures. We conduct extensive experiments quantitatively and qualitatively to verify that our FAN performs well on the real image super-resolution task of AIM 2020 challenge. According to the released final results, our team SR-IM achieves the fourth place on the X4 track with PSNR of 31.1735 and SSIM of 0.8728.
Entity Extraction of Electrical Equipment Malfunction Text by a Hybrid Natural Language Processing Algorithm

Many electrical equipment malfunction text messages are collected during power system operation and maintenance procedures.These texts usually contain crucial information for maintenance and condition monitoring.Because these power system malfunction texts are characterized by multidomain vocabularies, complex-syntactic structures, and long sentences, it is challenging to for automated systems to capture their semantic meaning and essential information.To address this issue, we propose a hybrid natural language processing (hybrid-NLP) algorithm to extract entities that represent electrical equipment.This algorithm is composed of a dictionary-based method, a language technology platform (LTP) tool, and the bidirectional encoder representations from a transformers-conditional random field (BERT-CRF) model.Significantly, the softmax output layer of the bidirectional encoder representations from the transformers (BERT) model is replaced by the conditional random field (CRF) algorithm to strengthen the contextual relationships between words and thus solve the local optimization of the preferred word label.The effectiveness of the proposed hybrid-NLP method is verified on a realistic dataset.Moreover, a statistical analysis is conducted to provide a reference for the operation and maintenance of power systems.
OrgBox

Current search systems are effective in helping users complete simple search tasks (e.g., fact-finding). However, they provide less support for users completing complex search tasks. Complex search tasks involve a diverse set of cognitive and metacognitive activities, such as goal-setting, organizing information, drawing inferences, monitoring progress, and updating mental models. We report on a lab study (N=32) that investigated the uses and influences of a novel knowledge representation tool called the "OrgBox'', developed to support searchers with complex tasks. The OrgBox was integrated into a custom-built search system and allowed participants to save information by drag-and-dropping textual passages into the tool, organize passages into "boxes'', and make notes on passages and boxes. The OrgBox tool was compared to a baseline tool (the "Bookmark'') that allowed participants to save passages, but not organize them nor make notes. We investigate four research questions. In RQ1, we investigate the effects of the knowledge representation tool on participants' post-task perceptions. In RQ2-RQ4, we investigate: (RQ2) how participants used different features of each tool; (RQ3) the perceived benefits and challenges of each tool; and (RQ4) the influences of each tool on the approaches taken by participants to complete the task. To address RQ2-RQ4, we conducted a qualitative analysis of participants' responses during an exit interview. We discuss implications from our results for designing tools to support users with complex search tasks.
Stacked Convolutional Sparse Auto-Encoders for Representation Learning

Deep learning seeks to achieve excellent performance for representation learning in image datasets. However, supervised deep learning models such as convolutional neural networks require a large number of labeled image data, which is intractable in applications, while unsupervised deep learning models like stacked denoising auto-encoder cannot employ label information. Meanwhile, the redundancy of image data incurs performance degradation on representation learning for aforementioned models. To address these problems, we propose a semi-supervised deep learning framework called stacked convolutional sparse auto-encoder, which can learn robust and sparse representations from image data with fewer labeled data records. More specifically, the framework is constructed by stacking layers. In each layer, higher layer feature representations are generated by features of lower layers in a convolutional way with kernels learned by a sparse auto-encoder. Meanwhile, to solve the data redundance problem, the algorithm of Reconstruction Independent Component Analysis is designed to train on patches for sphering the input data. The label information is encoded using a Softmax Regression model for semi-supervised learning. With this framework, higher level representations are learned by layers mapping from image data. It can boost the performance of the base subsequent classifiers such as support vector machines. Extensive experiments demonstrate the superior classification performance of our framework compared to several state-of-the-art representation learning methods.
Perceptual and computational detection of face morphing

A relatively new type of identity theft uses morphed facial images in identification documents in which images of two individuals are digitally blended to create an image that maintains a likeness to each of the original identities. We created a set of high-quality digital morphs from passport-style photos for a diverse set of people across gender, race, and age. We then examine people’s ability to detect facial morphing both in terms of determining if two side-by-side faces are of the same individual or not and in terms of identifying if a face is the result of digital morphing. We show that human participants struggle at both tasks. Even modern machine-learning-based facial recognition struggles to distinguish between an individual and their morphed version. We conclude with a hopeful note, describing a computational technique that holds some promise in recognizing that one facial image is a morphed version of another.
Dual Part-pooling Attentive Networks for Session-based Recommendation

Session-based recommendation is proposed to predict user preferences in a short anonymous interaction session. There are two kinds of characteristic in the session: sequential dependencies and collective dependencies. Sequential dependencies mean that user-item interactions in the session are strictly ordered, while collective dependencies mean that multiple interactions with a flexible order jointly determine the user's next behaviors. However, existing methods only take one of the two characteristics into account, which leads to the fact that they can't model user behavior well. Furthermore, few methods emphasize the first interaction in the session, although user's initial intention can be reflected by it to a large extent. To this end, we propose a novel model, Dual Part-pooling Attentive Networks for session-based Recommendation (DPAN4Rec), which is capable of capturing the sequential dependencies and collective dependencies of sessions simultaneously. By proposing a part-pooling attention mechanism, DPAN4Rec explores user's initial intention from first click and filters out noisy clicks in the session. Extensive experiments have been conducted on two benchmark e-commerce datasets, Yoochoose and Diginetica, and the experimental results show that DPAN4Rec outperforms state-of-the-art methods. Furthermore, our study demonstrates that it is necessary to consider both sequential dependencies and collective dependencies in session-based recommendation.
Research on an Image Denoising Algorithm based on Deep Network Learning

Abstract In recent years, the learning method based on depth convolution neural network has achieved unprecedented results in image denoising, and it has become a research hotspot to obtain better image denoising effect by adjusting network structure and parameters. Noise reduction convolution neural network adopts residual learning method in depth neural network, which not only improves the effect of noise reduction, but also solves the problem of blind noise reduction to a certain extent. Its disadvantage is that the algorithm has a long convergence time. In this paper, the structure of noise reduction convolution neural network is further improved, and an image denoising algorithm based on deconvolution neural network is proposed. The main features of this paper are as follows: 1) in the original network structure, deconvolution neural network is introduced to optimize the residual learning method; 2) A new loss function calculation method is proposed. BSD68 and SET12 test data sets are used to verify the proposed method. The experimental results show that the noise reduction performance of this algorithm is compared with the noise reduction convolution neural network algorithm. Under the same noise reduction effect, the convergence time of this algorithm is shortened by 120%. 138%. At the same time, compared with the traditional deep learning image denoising algorithm, the denoising effect and running efficiency of this method are also improved.
Indian Sign Language Interpretation and Sentence Formation

People with speech and hearing disabilities approximately constitute 1 percentage of the total Indian population. A person who is hearing and speech impaired is not able to compete or work with a normal person in a normal environment because of the lack of a proper communication medium. Sign Language is used for communication amongst them. Sign Language is the most natural and expressive way for the hearing and speech impaired. This paper proposes a method that recognizes Sign Language and converts it to normal text and speech for fast and improved communication amongst them and also with others. The focus is on the Indian Sign Language (ISL) specifically as there is no substantial work on ISL rendering the above requirements for these people. The paper focuses on developing a real-time hands-on system that takes video inputs of gestures in the specified ROI and performs gesture recognition using various feature extraction techniques and Hybrid-CNN model trained using the ISL database created. The correctly identified gesture tokens are sent to a Rule-Based Grammar and for Web Search query to generate various sentences and a Multi-Headed BERT grammar corrector provides grammatically precise and correct sentences as the final output.
Implementation of Grey Scale Normalization in Machine Learning &amp; Artificial Intelligence for Bioinformatics using Convolutional Neural Networks

Machine Learning is a trending field nowadays and is very well known as an application of Artificial Intelligence (AI). Machine learning makes use of secure arithmetical algorithms to construct computers assignment in a constructive way without being unequivocally programmed. The algorithms acquire freedom of a participated value and estimate output for this by utilizing definite arithmetical methods. The key function of machine learning is to create smart machines that can visualize and work related to human beings. Artificial Intelligence has been witnessing a massive improvement in bridging the gap among the capabilities of humans and technologies. Similarly, one of the characteristics of the field were tried to combine the outstanding effects. A Convolution Neural Network (CNN) is a sort of Deep Learning algorithm which can obtain an input image, assign consequence to different aspects in the image and capable to differentiate one from the other. The pre-processing requirement in a CNN is lesser as compared to other classification algorithms. CNN has the capability to find out these filter's characteristics. Bioinformatics is a term that is a mixture of two terms bio and informatics. Bio means associated with biology and informatics means in a sequence of information. Thus bioinformatics is an area that deals with handing out and accepting biological statistics using the computational and arithmetical approaches. Machine Learning has added up to of applications in the area of bioinformatics. Machine Learning finds its submission in the subsequent subfields of bioinformatics. The aim of this article is to perform a grayscale normalization of a selected image and thereafter to reduce the effect of illumination differences. Normalization is considered so that CNN works in a faster manner. Different models are available but Keras model is selected to perform this task. Keras supports the style of data preparation for Image statistics via the Image Data Generator group and Application Programming Interface (API). The Image Data Generator group in Keras provides a matching set of techniques for scaling pixel standards in the image dataset subsequent to modeling. The Keras functional API provides an additional flexible approach for significant models. It particularly allows Identifving several input or output models as well as models that allocate layers.
Efficient activation functions for embedded inference engines

The importance of the choice of the activation function for training and inferencing in machine learning cannot be overemphasized. Activation functions can influence network training convergence, performance accuracy, and can make training and inference stages computationally expensive. We introduce a family of square-based activation functions for embedded devices that consume only one instruction cycle with the potential of being resource efficient when constructed in silicon. We show that the proposed family are computationally efficient when compared with exponential-based non-linearities. The family includes functions for deep neural network architectures, support vector machines, recurrent neural networks, and others. We demonstrate the universal ability of the square-based family of activation functions on a variety of neural network architectures. We analyze the hidden representations of our trained multilayer perceptron network in an attempt to explain the performance gains, and speed-up observed when using square non-linearities. Speed-up was recorded with the quadratic-based kernel transformation on support vector machines. We record higher performance accuracy for recurrent neural network architectures and logistic regression using this family. Speed up was also recorded for implementation on Intel CPU and ARM processors. This family will find particular importance in low-end hardware devices with limited hardware capabilities.
Proposal of conversational chatbots for educational remediation in the context of covid-19

This paper presents a conversational chatbots solution allowing, in the context of covid-19, to support, detect and reduce deficiencies and bring students to self-training in order to develop their skills. The chatbot intervenes during formative evaluations and identifies the student's misunderstandings and directs him/her at the end of the test towards the parts of the course dealing with the concepts that have not been mastered. In the context of covid-19, one of the barrier gestures being social distancing, face-to-face classes have been suspended. Many higher education institutions that were not prepared for e-learning suddenly switched to online courses in order not to lose the academic year. Students no longer have the follow-up that they had when they were learning face-to-face. The chatbot proposed in this work could help reduce the negative effects of the covid-19 pandemic on the students' level. To allow learners to have a standardized learning environment, we have developed a plugin of the Moodle learning environment using the chatbot based on the Natural Language Processing (NLP) of Artificial Intelligence. In this work we show how our conversational chatbot is used for pedagogical remediation by indicating to the learner which chapters of the course to revise according to the gaps highlighted by the chatbot.
Lightweight Blockchain-based Platform for GDPR-Compliant Personal Data Management

New digital technologies generate large amounts of information. This data is processed by Service Providers in order to improve and develop new services or products, but also to fund themselves. However, processing these personal data can result in the extraction of sensitive information. In consequence, it can lead to users' privacy risk. To mitigate this risk, the EU elaborated the GDPR. It forces Service Providers to have Data Subjects' explicit consent for collecting and processing their personal data. The problem is that legislative text does not define how to transparently demonstrate that they already have this consent. Also, most users do not know the rights they have over their personal data, neither this regulation provides techniques for them to be aware about what happens with it. In this paper, we propose a lightweight blockchain-based GDPR-compliant personal data management platform. It provides public access to immutable evidences that show the agreements between the Data Subjects and Service Providers. The Service Providers can demonstrate that they are fulfilling the regulation, and Data Subjects are aware about what happens with their personal data and can manage it according to their rights.
Open-CV Social Distancing Intelligent System

Social distancing is a new norm that is effective and highly recommended in the pandemic situation and also now it is part of a day to day life, needing the individuals to stay a distance apart for varied activities to run smoothly. This system helps us ensure the proper social distancing in crowded places and highlights the violations of these norms in real-time. The system will take a live feed from established video recorders and will make a counter of violations eventually highlighting the areas of highest criticality on basis of the number of violations. The system works on a per-frame as bedrock enabling the highest accuracy, a frame is fetched the objects(humans) to be considered is detected, the distance between the detected objects are computed, and if this comes less than the acceptable intervention, it's marked as a violation. All this happens in real-time making it aberrant. Hence making the impact abated for a pandemic or intended activity due to while social distancing is enforced.