Reviewerly/exHarmony huggingface dataset:
Evaluating conversational recommender systems

Conversational recommender systems aim to interactively support online users in their information search and decision-making processes in an intuitive way. With the latest advances in voice-controlled devices, natural language processing, and AI in general, such systems received increased attention in recent years. Technically, conversational recommenders are usually complex multi-component applications and often consist of multiple machine learning models and a natural language user interface. Evaluating such a complex system in a holistic way can therefore be challenging, as it requires (i) the assessment of the quality of the different learning components, and (ii) the quality perception of the system as a whole by users. Thus, a mixed methods approach is often required, which may combine objective (computational) and subjective (perception-oriented) evaluation techniques. In this paper, we review common evaluation approaches for conversational recommender systems, identify possible limitations, and outline future directions towards more holistic evaluation practices.
Saliency-Guided Local Full-Reference Image Quality Assessment

Research and development of image quality assessment (IQA) algorithms have been in the focus of the computer vision and image processing community for decades. The intent of IQA methods is to estimate the perceptual quality of digital images correlating as high as possible with human judgements. Full-reference image quality assessment algorithms, which have full access to the distortion-free images, usually contain two phases: local image quality estimation and pooling. Previous works have utilized visual saliency in the final pooling stage. In addition to this, visual saliency was utilized as weights in the weighted averaging of local image quality scores, emphasizing image regions that are salient to human observers. In contrast to this common practice, visual saliency is applied in the computation of local image quality in this study, based on the observation that local image quality is determined both by local image degradation and visual saliency simultaneously. Experimental results on KADID-10k, TID2013, TID2008, and CSIQ have shown that the proposed method was able to improve the state-of-the-art’s performance at low computational costs.
A comprehensive empirical study on bug characteristics of deep learning frameworks

Deep Learning (DL) frameworks enable developers to build DNN models without learning the underlying algorithms and models. While some of these DL-based software systems have been deployed in safety-critical areas, such as self-driving cars and medical diagnostics, for DL frameworks, characterizing their bugs and thus helping researchers to design specific quality assurance techniques become desperately needed. Our research aims to characterize bugs typical of DL frameworks at the source code level for an in-depth analysis of bug symptoms, root causes, and bug fixes. In this way, we hope to provide insights for researchers to design automatic quality assurance techniques, such as automatic repair techniques and fault location techniques, applicable to DL frameworks and DL-based software systems. We started by summarizing the DL framework reference architecture and proposing the DL framework bug taxonomy. Then, we mined 1,127 DL framework bug reports from eight popular DL frameworks and labeled the bug types, root causes, and symptoms. Finally, we discussed the bug characteristics and explored how developers could possibly deal with these bugs. Our main findings are: (i) DNN model building bugs and general type bugs accounted for one-third of the total defects. (ii) DNN model building bugs are more prone to algorithm logic constraints, internal API errors, and data/numerical errors. (iii) Fifteen bug-fixing patterns are summarized, providing reference for common DL framework bug repair and future research on the development of automatic DL framework bug detection tools. By analyzing the bug-fixing changes, we characterize the occurrences, root causes, symptoms, and fixing of these bugs. The study results have provided researchers with insights into how to ensure DL framework quality and presented actionable suggestions for DL framework developers to improve their code quality.
Natural Language Processing for Automated Classification of Qualitative Data From Interviews of Patients With Cancer

This study sought to explore the use of novel natural language processing (NLP) methods for classifying unstructured, qualitative textual data from interviews of patients with cancer to identify patient-reported symptoms and impacts on quality of life.We tested the ability of 4 NLP models to accurately classify text from interview transcripts as "symptom," "quality of life impact," and "other." Interview data sets from patients with hepatocellular carcinoma (HCC) (n = 25), biliary tract cancer (BTC) (n = 23), and gastric cancer (n = 24) were used. Models were cross-validated with transcript subsets designated for training, validation, and testing. Multiclass classification performance of the 4 models was evaluated at paragraph and sentence level using the HCC testing data set and analyzed by the one-versus-rest technique quantified by the receiver operating characteristic area under the curve (ROC AUC) score.NLP models accurately classified multiclass text from patient interviews. The Bidirectional Encoder Representations from Transformers model generally outperformed all other models at paragraph and sentence level. The highest predictive performance of the Bidirectional Encoder Representations from Transformers model was observed using the HCC data set to train and BTC data set to test (mean ROC AUC, 0.940 [SD 0.028]), with similarly high predictive performance using balanced and imbalanced training data sets from BTC and gastric cancer populations.NLP models were accurate in predicting multiclass classification of text from interviews of patients with cancer, with most surpassing 0.9 ROC AUC at paragraph level. NLP may be a useful tool for scaling up processing of patient interviews in clinical studies and, thus, could serve to facilitate patient input into drug development and improving patient care.
A Small Object Tracking Method in Satellite Videos Based on Improved Kernel Correlation Filter

The video satellite enjoys the ability that it can stare at the interesting area and record object information in the form of continuous video sequences, thus making it have significant advantages in change detection, object tracking and other ap-plications. However, objects in satellite videos generally account for a small proportion of the images and are easily affected by occlusion. As a result, most of the current algorithms can not achieve satisfactory results in satellite videos object tracking. To solve these problems, this paper proposes a satellite video object tracking method based on improved kernel correlation filter. The main innovations of our method are as follows: 1) The average peak correlation energy and peak value of the response map are used to judge whether the object is occluded or not, which provide the criterion for adjusting the update policy of correlation filter template; 2) The Kalman filter is utilized to estimate the motion status of the object and accurately predict the position when the object is occluded. Based on the improvement, the proposed method effectively solves the problem of tracking failure when the moving object is partially or completely occluded. The experimental results show that the method can track the moving objects in satellite videos with 97.7% accuracy, and the speed reaches 2000 frames per second.
Implementation of Semantic Textual Similarity between Requirement Specification and Use Case Description Using WUP Method (Case Study: Sipjabs Application)

The SRS used in this study is an application called “Sipjabs”. This application processes data regarding the position of human resources to meet the needs of a company. This research aims to implement semantic textual similarity in software requirements specification through functional requirements with use case diagrams using the Wu Palmer (WUP) method in finding semantics. This research method is presented in a flow chart consisting of three main activities: research object analysis, semantic textual similarity, and validity and reliability testing. In this research, an extraction process for the Requirement Specification has been produced, divided into five documents: FR01, FR02, FR03, FR04, FR05. Then the steps performed in the use case description are divided into UD01, UD02, UD03, UD04, UD05. The highest similarity value is found in documents UD03 and FR03, where the number of similarities is 0.626640. In addition, the highest score of the sentence that has been calculated using the Wu Palmer concept is 0.8000, which is found in the words “page” and “user”. The highest kappa value with Gwet's AC1 formula is 0.02547770700636931, which means “Fair Agreement”. For the results of the calculation of the questionnaire filled in by the expert, namely 0.82022, which means “Almost Perfect”.
Leveraging Transfer Learning and GAN Models for OCR from Engineering Documents

Digital engineering, the digital transformation of engineering practice, is profoundly changing the traditional engineering practice towards the fast integration of digital technologies and digital models in the engineering processes' life cycles. The traditional engineering process heavily relies on static engineering documents (e.g., spreadsheets, technical drawings, and scanned documents) to store and share information across the engineering process. A critical task in digital engineering is to extract relevant textual information from traditional engineering documents into machine-readable and editable formats. This paper explores deep learning models and OCR methods to effectively extract textual information from engineering documents collected by the NAVY's military sealift command division. We propose a deep learning-based optical character recognition (OCR) framework for this task, which integrates several modules including a pre-trained text detection model, a fine-tuned OCR algorithm, and a deep generative model to augment data for the fine-tuning. Experimental results showed that the fine-tuning method significantly improved word accuracies of OCR models from 60%-70% to 90% and above. Furthermore, the deep adversarial generative approach had proved to be an effective model for data augmentation.
Implicit LiDAR Network: LiDAR Super-Resolution via Interpolation Weight Prediction

Super-resolution of LiDAR range images is crucial to improving many downstream tasks such as object detection, recognition, and tracking. While deep learning has made a remarkable advances in super-resolution techniques, typical convolutional architectures limit upscaling factors to specific output resolutions in training. Recent work has shown that a continuous representation of an image and learning its implicit function enable almost limitless upscaling. However, the detailed approach, predicting values (depths) for neighbor pixels in the input and then linearly interpolating them, does not best fit the LiDAR range images since it does not fill the unmeasured details but creates a new image with regression in a high-dimensional space. In addition, the linear interpolation blurs sharp edges providing important boundary information of objects in 3-D points. To handle these problems, we propose a novel network, Implicit LiDAR Network (ILN), which learns not the values per pixels but weights in the interpolation so that the super-resolution can be done by blending the input pixel depths but with non-linear weights. Also, the weights can be considered as attentions from the query to the neighbor pixels, and thus an attention module in the recent Transformer architecture can be leveraged. Our experiments with a novel large-scale synthetic dataset demonstrate that the proposed network reconstructs more accurately than the state-of-the-art methods, achieving much faster convergence in training.
Multi-UAV Disaster Environment Coverage Planning with Limited-Endurance

Disaster areas involving floods and earthquakes are commonly large, with the rescue time being quite tight, suggesting multi-Unmanned Aerial Vehicles (UAV) exploration rather than employing a single UAV. For such scenarios, current UAV exploration is modeled as a Coverage Path Planning (CPP) problem to achieve full area coverage in the presence of obstacles. However, the UAV's endurance capability is limited, and the rescue time is constrained, prohibiting even multiple UAVs from completing disaster area coverage on time. Therefore, this paper defines a multi-Agent Endurance-limited CPP (MAEl-CPP) problem that is based on an a priori known heatmap of the disaster area, which affords to explore the most valuable areas under UAV limited energy constraints. Furthermore, we propose a path planning algorithm for the MAEl-CPP problem by ranking the possible disaster areas according to their importance through satellite or remote sensing aerial images and completing path planning according to this ranking. Experimental results demonstrate that the search efficiency of the proposed algorithm is 4.2 times that of the existing algorithm.
Privacy-Enhancing Settlements Protocol in Peer-to-Peer Energy Trading Markets

This paper proposes a privacy-friendly billing and settlements system for use in peer-to-peer energy trading markets. More specifically, it proposes a privacy-enhancing settlements protocol (PESP) which makes use of a partial homomorphic encryption scheme to facilitate private processing and computation of smart grid users’ monthly energy bill by a semi-trusted third party. In addition, it allows for minor financial penalties to users who participate in peer-to-peer trading but do not accurately predict their energy usage for each period and, as such, import or export varying amounts compared to the amount they have agreed to trade for initially. This encourages predictions to be accurate to help keep the trading fair and aiding suppliers to make accurate predictions about the amount of electricity needed for each trading period. Our security analysis and performance evaluation demonstrate that PESP protects users’ privacy and is efficient in terms of computation and communication cost.
A Practical Deceptive Jamming Method Based on Vulnerable Location Awareness Adversarial Attack for Radar HRRP Target Recognition

In recent years, deep neural networks are increasingly popular in the field of radar high-resolution range profiles (HRRPs) target recognition. Unfortunately, recent researches have revealed that a deep-learning classifier can be easily fooled by adding small perturbations to the input, named adversarial attack. This provides us an inspiration for radar deceptive jamming signal generation in electronic countermeasures (ECMs). However, the perturbations generated by these adversarial attacks are usually of complex envelopes and quite low power, making it challenging for jammers to generate such actual jamming signals. To solve that issue, we propose a practical deceptive jamming generation method that learns the vulnerable range cells in an HRRP sample and injects several jamming pulses with specific amplitudes into these range cells. Such jamming signals are easy to generate and can deceive the radar automatic target recognition (RATR) model to output the wrong target category prediction with high confidence. To avoid the requirement of the recognition network structure information, we leverage the differential evolution optimization algorithm (non-gradientbased). Further, to provide the potential of real-time jamming signal generation during the test, an encoder is constructed not only to learn the separable features but also to find the vulnerable range cells and the specific amplitudes of the jamming pulses. In the experiments, we apply the proposed attack algorithms to fool the one-dimensional convolutional neural network-based HRRPRATR models. The extensive experimental results on measured aircraft HRRP dataset prove that the proposed algorithms achieve a promising attack performance and serve as a practical and fast deceptive jamming generation method.
Fully Convolutional Encoder-Decoder With an Attention Mechanism for Practical Pedestrian Trajectory Prediction

Pedestrian trajectory prediction using video is essential for many practical traffic applications. Most existing pedestrian trajectory prediction methods are based on fully connected long short-term memory (LSTM) networks and perform well on public datasets. However, these methods still have three defects: a) Most of them rely on manual annotations to obtain information about the environment surrounding the subject pedestrian, which limits practical applications; b) The interaction among pedestrians and obstacles in a scene is little studied, which leads to greater prediction error; c) Traditional LSTM methods are based on the previous moment and ignore the correlation between the future and distant past states of the pedestrian, which generates unrealistic trajectories. To tackle these problems, first, in the stage of data processing, we use an image semantic segmentation algorithm to obtain multi-category obstacle information and design an end-to-end “Siamese Position Extraction” model to obtain more accurate pedestrian interaction data. Second, we design an end-to-end fully convolutional LSTM encoder-decoder with an attention mechanism (FLEAM) to overcome the shortcomings of LSTM. Third, we compare FLEAM with several state-of-the-art LSTM-based prediction methods on multiple video sequences in the datasets ETH, UCY and MOT20. The results show that our approach generates the same prediction error as the best results of the state-of-the-art method. However, FLEAM has more potential for practice application because it does not rely on manually annotated data. We further validate the effectiveness of FLEAM by employing manually annotated data, finding that it generates much less prediction error.
DLI-Net: Dual Local Interaction Network for Fine-Grained Sketch-Based Image Retrieval

Fine-grained sketch-based image retrieval (FG-SBIR) is considered an ideal method of image retrieval due to the rich and easily accessible characteristics of sketches. It aims to find the most similar photo from the photo gallery based on the input sketch. Most previous works follow the paradigm that extracting global feature first and then projecting the features of sketch and photo to unified embedding feature space using triplet loss. However, the global feature is not appropriate for extracting the crucial fine-grained information. Based on this principle, we propose a Dual Local Interaction Network (DLI-Net). DLI-Net explores an effective and efficient way to utilize local features for FG-SBIR. Specifically, we first propose a Local Feature Extractor to extract mid-level local features. Then, in response to the problems brought by local features, we propose a Dual Interaction Module, which contains Self Interaction Module and Cross Interaction Module. Self Interaction Module speeds up retrieval by eliminating the redundant local features of background. Cross Interaction Module solves the spatial misalignment by making the sketches interact with photos. Extensive experiments on six commonly used datasets show that our DLI-Net outperforms state-of-the-art competitors by a significant margin with a reasonable retrieval speed. Moreover, to the best of our knowledge, DLI-Net is the first model that beats humans on all six datasets. Besides, DLI-Net also performs best on cross-category fine-grained sketch-based image retrieval task, which further demonstrates local features are more appropriate for FG-SBIR.
Motion Recognition in Bharatanatyam Dance

This paper provides a method to understand the underlying semantics of Bharatnatyam dance motion and classifies it.Each dance performance is audio-driven and spans over space and time.The dance is captured and analyzed, which is helpful in cultural heritage preservation, and tutoring systems to assist the naive learner.This paper attempts to solve the fundamental problem; recognizing the motions during a dance performance based on motion-pattern.The used dataset is the video recordings of an Indian Classical Dance form known as Bharatanatyam.The different Adavus (The basic unit of Bharatanatyam) of Bharatanatyam dance are captured using Kinect.We choose RGB from various forms of captured data (RGB, Depth, and Skeleton).Motion History Image (MHI) and Histogram of Gradient of MHI (HoGMHI) are computed for each motion and used as an input for the Machine Learning (ML) algorithms to recognize motion.The paper explores two ML techniques; Support Vector Machine (SVM) and Convolutional Neural Network (CNN).The overall accuracy of both the classifiers is more than 90%.The novelties of the work are (a) analysing all possible involved motions based on the motion-patterns rather than the joint velocities or pose, (b)exploring the impact of training data and the different features on the classifiers' accuracy, (c) not restricting the number of frames in a motion during recognition and formulate a method to deal with the variable number of frames in the motions.
Integration of Mixed Reality to CFD in Industry 4.0: A Manufacturing Design Paradigm

Simulation, as a pillar technology of Industry 4.0, is a key tool for improving manufacturing processes. Simulation is an indispensable set of technological tools and methods for the successful implementation of digital manufacturing, since it allows for the experimentation and validation of product, process, and system design and configuration. The value of simulation is evident, especially in today’s volatile manufacturing environment, which is influenced by globalization and ever-increasing demands for higher levels of product customization and personalization. Moreover, a significant manufacturing issue is the interoperability and communication among the several simulation tools. To that end, an Industrial Internet of Things (IIoT) framework for simulation and visualization of selected monitored values based on Mixed Reality (MR) and Computational Fluid Dynamics (CFD) in a cloud-computing environment for intuitive data visualization on smart devices is presented in this paper. As such, in light of the possibilities offered by digital technologies such as CFD simulation, smart factories can gain a competitive advantage because of the low cost, low risk, and the provided useful insights. The range of CFD applications encountered in process manufacturing and engineering varies greatly depending on the industry. Summarizing, this review paper identifies the gaps in current practices, and future trends and challenges to be met on the field are outlined. The contribution of the research work extends to the proposal of a workflow for the integration of Mixed Reality (MR) to the CFD simulation, in an attempt to create simulation holograms on the user’s field of view.