Reviewerly/exHarmony huggingface dataset:
Multi-hop Question Answering with Knowledge Graph Embedding in a Similar Semantic Space

Multi-hop Question Answering using the knowledge graph (KG) as a data source requires subject entities and relations that are obtained from natural language questions; the answers are then obtained by reasoning through multiple triples in the KG. However, even large KGs are incomplete, and the reasoning often fails to obtain the correct answer due to the lack of relations in the KG. Recently, researchers have proposed introducing KG embedding into the multi-hop knowledge graph question answering (KGQA) field to solve the incompleteness of the KG. However, these methods embed the question and the KG into different semantic spaces, and it is difficult to obtain the correct answers. Furthermore, due to the limitation of the question embedding sequence, the contribution of each word to the question semantics cannot be distinguished. To overcome the above problems, this paper proposes an effective multi-hop KGQA model, TIPNet, using relation embeddings in knowledge graph triples, which uses the idea of translation models to narrow the semantic spatial distance between question embeddings and KG embeddings. At the same time, TIP weighting technology is proposed to distinguish the semantic contribution of words to the question. To validate the performance of TIPNet, experiments were conducted on the WebQSP and CWQ datasets, and the model reached advanced levels under both KG-full and KG-half settings.
Adversarial Erasing Framework via Triplet with Gated Pyramid Pooling Layer for Weakly Supervised Semantic Segmentation

Weakly supervised semantic segmentation (WSSS) has employed Class Activation Maps (CAMs) to localize the objects. However, the CAMs typically do not fit along the object boundaries and highlight only the most-discriminative regions. To resolve the problems, we propose a Gated Pyramid Pooling (GPP) layer which is a substitute for a Global Average Pooling (GAP) layer, and an Adversarial Erasing Framework via Triplet (AEFT). In the GPP layer, a feature pyramid is obtained by pooling the CAMs at multiple spatial resolutions, and then be aggregated into an attention for class prediction by gated convolution. With the process, CAMs are trained not only to capture the global context but also to preserve fine-details from the image. Meanwhile, the AEFT targets an over-expansion, a chronic problem of Adversarial Erasing (AE). Although AE methods expand CAMs by erasing the discriminative regions, they usually suffer from the over-expansion due to an absence of guidelines on when to stop erasing. We experimentally verify that the over-expansion is due to rigid classification, and metric learning can be a flexible remedy for it. AEFT is devised to learn the concept of erasing with the triplet loss between the input image, erased image, and negatively sampled image. With the GPP and AEFT, we achieve new state-of-the-art both on the PASCAL VOC 2012 val/test and MS-COCO 2014 val set by 70.9%/71.7% and 44.8% in mIoU, respectively.
Challenges in managing semantic annotations in harvested research objects in a national CRIS context

Harvested metadata on research objects can include links between the primary domain objects such as organizational identifiers associated with dataset, persons identified with ORCIDs linked to publications and publications connected through ISSNs to publishing channels. This kind of linkage is the bread-and-butter of the CRIS systems and usually comprehensively maintained. When it comes to the more subjective description of a domain object, such as keywords, themes, or subject headings, the issues related to data management and modeling become prominent with challenges such as flexibility of free text keywords as opposed to authoritative, but rigid classification systems. Many CRIS objects also already contain an extensive description of the content, just meant for human consumption, in the form of an abstract or similar summary text.With the help of automated data mining and annotation tools, these textual representations can be processed into structured data. This paper presents the processing pipelines implemented as part of the research.fi portal for automatic linking of different research inputs based on automatically extracted ontology concepts and discusses the implications of utilizing them as part of the research.fi platform. But more than simply discussing the annotation of research objects and the creation of word clusters for representation of the semantic content of research objects, we also discuss challenges related to maintaining the automatically produced metadata
Context change and triggers for human intention recognition

In human-robot interaction, understanding human intention is important to smooth interaction between humans and robots. Proactive human-robot interactions are the trend. They rely on recognising human intentions to complete tasks. The reasoning is accomplished based on the current human state, environment and context, and human intention recognition and prediction. Many factors may affect human intention, including clues which are difficult to recognise directly from the action but may be perceived from the change in the environment or context. The changes that affect human intention are the triggers and serve as strong evidence for identifying human intention. Therefore, detecting such changes and identifying such triggers are the promising approach to assist in human intention recognition. This paper discusses the current state of art in human intention recognition in human-computer interaction and illustrates the importance of context change and triggers for human intention recognition in a variety of examples.
A Method of Ambiguity Detection in Requirement Specifications by Using a Knowledge Dictionary

Early detection of ambiguities in requirement specifications is effective in reducing the occurrence of specification changes and construction period delays in later processes. However, it is not easy to detect ambiguities in requirement specifications written in natural language without knowledge or experience. In this paper, for detecting ambiguities in requirement specifications, we propose a method of using a knowledge dictionary constructed by focusing on transitive verbs and their objects in requirement specifications and documents that confirm their contents.
A Runtime Programmable Accelerator for Convolutional and Multilayer Perceptron Neural Networks on FPGA

Deep neural networks (DNNs) are prevalent for many applications related to classification, prediction and regression. To perform different applications with better performance and accuracy, an optimized network architecture is required, which can be obtained through experiments and performance evaluation on different network topologies. However, a custom hardware accelerator is not scalable and it lacks the flexibility to switch from one topology to another at run time. In order to support convolutional neural networks (CNN) along with multilayer perceptron neural networks (MLPNN) of different sizes, we present in this paper an accelerator architecture for FPGAs that can be programmed during run time. This combined CNN and MLP accelerator (CNN-MLPA) can run any CNN and MLPNN applications without re-synthesis. Therefore, time spent on synthesis, placement and routing can be saved for executing different applications on the proposed architecture. Run time results show that the CNN-MLPA can be used for network topologies of different sizes without much degradation of performance. We evaluated the resource utilization and execution time on Xilinx Virtex 7 FPGA board for different benchmark datasets to demonstrate that our design is run time programmable, portable and scalable for any FPGA. The accelerator was then optimized to increase the throughput by applying pipelining and concurrency, and reduce resource consumption with fixed-point operations.
Indian Sign Language to Speech Conversion Using Convolutional Neural Network

True incapacity is the inability to speak. A person who has a speech impediment is unable to communicate with others through speech and hearing. Individuals use sign language as a form of communication to overcome this disability. Even though signing has become commonplace in recent years, it can still be difficult for non-signers to communicate with signers. The flow of information and emotions in a person’s life has become increasingly dependent on communication over time. The only way for that person with special needs to communicate with the rest of the world is through sign language, which uses entirely distinct hand motions. With the most recent developments in computer vision and deep learning techniques, there has been significant improvement in the disciplines of motion and gesture identification. For American Sign Language (ASL), sign language recognition has been a well-researched subject. Nevertheless, there aren’t much published analysis works on Indian Sign Language (ISL). The intended method will recognise 4972 static hand signs for the twenty-four different English alphabets (A, B, C, D, E, F, G, H, I, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y). The main goal of our effort is to create a deep learning-based application that uses the "Google text to speech" API to translate sign language into text, facilitating communication between signers and non-signers. We made use of the Kaggle-available dataset. Proposed method using custom Convolutional Neural Network and got the accuracy of 99%.
Examination of e-commerce studies between 2016 and 2021 with science mapping and network analysis

AbstractThe concept of trade is constantly changing and getting stronger with innovations. Today, with these innovations, the concept of e-commerce comes to the fore and increases its popularity day by day. E-commerce is also known as electronic commerce or electronic marketing. E-commerce briefly means buying or selling products or services via the internet. In this way, small and medium enterprises (SMEs) can reach more consumers or customers. This causes the development of e-commerce to be even faster. In this study, 2798 academic publications about e-commerce were reviewed to define e-commerce trends, bibliographic couplings of authors, journals and countries, and most used terms in the field yearly. In this context, firstly, annual changes in the number of publications are given. Then, the change in research areas was examined on an annual basis and it was stated which areas came to the fore. Then, detailed analyzes (trends and bibliographic couplings) about authors, journals and countries were included. In the end, term extraction process was conducted on a yearly basis. Scientific mapping of the field, which was obtained by the term extraction process, is given and interpreted.Keywords: E-commerceBibliographic CouplingTerm extractionTrends
Deep Learning For Channel Estimation And Signal Detection

Channel estimation and signal detection are essential steps in communication systems, they are used along with different modulation techniques such as orthogonal frequency-division multiplexing (OFDM) to ensure the success of the system as whole. There are some conventional methods such as least square (LS) that have been widely used in channel estimation for OFDM systems, the problem with LS is that it is not accurate enough especially in low signal-to-noise ratio (SNR) regions. Deep learning (DL) can be a possible solution to solve many problems in wireless communications. This project proposes a DL model for channel estimation and signal detection in OFDM systems. We adopted a deep neural network (DNN) model to provide an effective solution to increase the accuracy with minimum complexity. The performance of the DNN model is evaluated and then compared with LS method simulations in terms of BER under different SNRs. The comparison shows that DL proves to have a better performance compared with LS simulations.
Electronic notes on general physics for a technical university

The problem and the goal. The article deals with the problem of creating electronic abstracts on general physics for a technical university as components of the electronic educational environment of universities within the framework of the global digitalization of higher education. The purpose of the article: to create and substantiate electronic summaries for each topic of the general physics course, structured in accordance with the main stages of the process of scientific cognition (live contemplation – abstract thinking– practice), corresponding to the criteria of completeness, clarity, applied and ideological significance. Methodology. The study used the following methods: theoretical analysis and generalization of russian and foreign literature on the problem of structuring the content of the general physics course at a technical university; modeling the content of physical education in accordance with the stages of the cognition process; experimental research methods to verify the effectiveness of created electronic notes, mathematical methods of processing the results of pedagogical experiment. Results. On the basis of the main stages of the cognition process, the structure of the electronic abstract is substantiated, consisting of five points: the study of basic physical concepts; visualization of physical concepts; the study of physical laws; visualization of physical laws; clarification of the meaning of concepts and laws in science and practice (ideological and applied). In accordance with this structure, electronic notes have been developed for all sections of the general physics course. The effectiveness of the developed electronic notes for the assimilation of the content of the general physics course in a pedagogical experiment using the statistics of the criterion of signs is proved. As a result of the conducted educational experiment it was possible to establish that the use of electronic notes leads to the formation of students' necessary knowledge of general physics (z = -2.581; p &lt; 0.01). Conclusion. In conclusion, it is concluded that the created electronic abstracts on general physics for a technical university, the structure of which reflects the stages of the cognition process, and the content, along with physical knowledge and their visualization, help students effectively master the course of general physics due to the properties of completeness, clarity and the establishment of applied and ideological significance of the topic.
Rethinking Feature Extraction: Gradient-Based Localized Feature Extraction for End-To-End Surgical Downstream Tasks

Several approaches have been introduced to understand surgical scenes through downstream tasks like captioning and surgical scene graph generation. However, most of them heavily rely on an independent object detector and region-based feature extractor. Encompassing computationally expensive detection and feature extraction models, these multi-stage methods suffer from slow inference speed, making them less suitable for real-time surgical applications. The performance of the downstream tasks also degrades from inheriting errors of the earlier modules of the pipeline. This work develops a detector-free gradient-based localized feature extraction approach that enables end-to-end model training for downstream surgical tasks such as report generation and tool-tissue interaction graph prediction. We eliminate the need for object detection or region proposal and feature extraction networks by extracting the features of interest from the discriminative regions in the feature map of the classification models. Here, the discriminative regions are localized using gradient-based localization techniques (e.g. Grad-CAM). We show that our proposed approaches enable the real-time deployment of end-to-end models for surgical downstream tasks. We extensively validate our approach on two surgical tasks: captioning and scene graph generation. The results prove that our gradient-based localized feature extraction methods effectively substitute the detector and feature extractor networks, allowing end-to-end model development with faster inference speed, essential for real-time surgical scene understanding tasks.
Building an Efficient Retrieval-based Dialogue System with Contrastive Learning

We focus on retrieval-based dialogue systems. Such a system aims to select an appropriate response from a candidate pool for a given context. Recent methods commonly utilize powerful interaction-based pre-trained language models like BERT to achieve the goal. However, their time cost is usually not satisfying since the procedure of computing relevance scores is not efficient, especially in scenarios that require online response selection. We propose an efficient dialogue system that utilizes a representation-based BERT to address this issue, which can produce an independent representation for every response candidate and context. The relevance score can be simply calculated by the dot product. We further enhance the representation ability of this model by applying domain adaptive post-training and supervised contrastive learning fine-tuning. Experimental results on two benchmark datasets show that our method achieves competitive performance with other interaction-based models while retaining the advantage of time efficiency. We also provide an empirical and theoretical analysis of time efficiency between representation-based models and interaction-based models. The main contribution of this paper is to propose a novel methodology to build a simple but efficient dialogue system.
C$^{2}$MT: A Credible and Class-Aware Multi-Task Transformer for SR-IQA

In this letter a novel credible and class-aware multi-task transformer abbreviated as C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> MT for SRIQA, is proposed. In the proposed C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> MT, a quality-aware task for the quality prediction and the other class-aware task for the classification of SR algorithms are jointly framed to mine mutual information between the quality of SR images and the class of SR algorithms for more discriminative perceptual representation. In the class-aware task, we develop a supervised contrastive learning strategy to learn embedding perceived features related to a class-specific SR algorithm. While in the other quality-aware task, we employ a novel credible pseudo quality label generation strategy to actively adjust the quality labels by ranking the pair-wise consistency between the predicted quality scores and subjective perceptual scores but keep the image-level quality labels unchanged. The developed supervised contrastive learning and the variant of active learning strategies benefit learning a more consistent quality predictor for SR images. Experiment results indicate that our proposed C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> MT achieves state-of-the-art results on five popular SRIQA benchmark databases.
Privacy-Preserving Object Detection With Poisoning Recognition for Autonomous Vehicles

Object detection has achieved significant progress in attaining high-quality performance without leaking private messages. However, traditional approaches cannot defend the poisoning attacks. Poisoning attacks can make the predictive model unusable, which quickly causes recognition errors or even traffic accidents. In this paper, we propose a privacy-preserving object detection with poisoning recognition (PR-PPOD) framework via distributed training with the help of the CNN, ResNet18, and classical SSD network. Specifically, we design a poisoning model recognition algorithm to remove the uploaded local poisoning parameters to guarantee a trained model's availability based on given privacy-preserving progress. More importantly, the PR-PPOD framework can effectively prevent the threat of differential attacks and avoid privacy leakage caused by reverse model reasoning. Moreover, the effectiveness, efficiency, and security of PR-PPOD are demonstrated via comprehensive theoretical analysis. Finally, we simulate the performance of local poisoning model recognition based on the MNIST, CIFAR10, VOC2007, and VOC2012 datasets, which could achieve good performance compared with the case without poisoning recognition.
Ensemble Recurrent Neural Networks and Their Optimization by Particle Swarm for Complex Time Series Prediction

In this article, we combine convolutional neural networks and particle swarm optimization techniques to design Recurrent Neural Network (RNN) architectures. The proposed particle swarm optimization seeks to find the number of layers, number of modules and number of neurons for layers. This method is applied to time series, the databases used are that of the Petroleum, The US Dollar/MX Exchange Rate and Taiwan Exchange Rate, where the main objective of this work is to find by means optimization algorithm is to minimize the perdition error for each of time series. The simulation results show that the recurrent neural network approach produces a good prediction error.
$$\mathrm {D^2ADA}$$: Dynamic Density-Aware Active Domain Adaptation for Semantic Segmentation

In the field of domain adaptation, a trade-off exists between the model performance and the number of target domain annotations. Active learning, maximizing model performance with few informative labeled data, comes in handy for such a scenario. In this work, we present $$\mathbf {D^2ADA}$$ , a general active domain adaptation framework for semantic segmentation. To adapt the model to the target domain with minimum queried labels, we propose acquiring labels of the samples with high probability density in the target domain yet with low probability density in the source domain, complementary to the existing source domain labeled data. To further facilitate labeling efficiency, we design a dynamic scheduling policy to adjust the labeling budgets between domain exploration and model uncertainty over time. Extensive experiments show that our method outperforms existing active learning and domain adaptation baselines on two benchmarks, GTA5 $$\rightarrow $$ Cityscapes and SYNTHIA $$\rightarrow $$ Cityscapes. With less than 5% target domain annotations, our method reaches comparable results with that of full supervision.
Continuous Engagement Estimation based on Gaze Estimation and Facial Expression Recognition

Existing engagement estimation methods only show discrete estimation results according to the degree of engagement, and do not express fine-grained state information of the object. In this paper, we propose a continuous engagement estimation framework using estimated gaze direction and facial expression information. Also, we build a video-based engagement estimation database and succeeded in qualitatively verifying the performance of our framework.