Reviewerly/exHarmony huggingface dataset:
CoSPARSE: A Software and Hardware Reconfigurable SpMV Framework for Graph Analytics

Sparse matrix-vector multiplication (SpMV) is a critical building block for iterative graph analytics algorithms. Typically, such algorithms have a varying active vertex set across iterations. This variability has been used to improve performance by either dynamically switching algorithms between iterations (software) or designing custom accelerators (hardware) for graph analytics algorithms. In this work, we propose a novel framework, CoSPARSE, that employs hardware and software reconfiguration as a synergistic solution to accelerate SpMV-based graph analytics algorithms. Building on previously proposed general-purpose reconfigurable hardware, we implement CoSPARSE as a software layer, abstracting the hardware as a specialized SpMV accelerator. CoSPARSE dynamically selects software and hardware configurations for each iteration and achieves a maximum speedup of 2.0 × compared to the naïve implementation with no reconfiguration. Across a suite of graph algorithms, CoSPARSE outperforms a state-of-the-art shared memory framework, Ligra, on a Xeon CPU with up to 3.51 × better performance and 877 × better energy efficiency.
Sequential credit card fraud detection: A joint deep neural network and probabilistic graphical model approach

Abstract With the wide usage of e‐banking in recent years, and by increased opportunities for fraudsters subsequently, we are witnessing a loss of billions of Euros worldwide due to credit card fraud every year. Therefore, credit card fraud detection has become a critical necessity for financial institutions. Several studies have used machine learning techniques for proposing a method to address the problem. However, most of them did not take into account the sequential nature of transactional data. In this paper, we proposed a novel credit card fraud detection model using sequence labelling based on both deep neural networks and probabilistic graphical models (PGM). Then by using two real‐world datasets, we compared our model with the baseline model and examined how considering hidden sequential dependencies among transactions and also among predicted labels can improve the results. Moreover, we introduce a novel undersampling algorithm, which helps to maintain the sequential patterns of data during the random undersampling process. Our experiments demonstrate that this algorithm achieves promising results compared to the state‐of‐the‐art methods in oversampling and undersampling.
A novel deep learning framework for double JPEG compression detection of small size blocks

Double JPEG compression detection plays a vital role in multimedia forensics, to find out whether a JPEG image is authentic or manipulated. However, it still remains to be a challenging task in the case when the quality factor of the first compression is much higher than that of the second compression, as well as in the case when the targeted image blocks are quite small. In this work, we present a novel end-to-end deep learning framework taking raw DCT coefficients as input to distinguish between single and double compressed images, which performs superior in the above two cases. Our proposed framework can be divided into two stages. In the first stage, we adopt an auxiliary DCT layer with sixty-four 8 × 8 DCT kernels. Using a specific layer to extract DCT coefficients instead of extracting them directly from JPEG bitstream allows our proposed framework to work even if the double compressed images are stored in spatial domain, e.g. in PGM, TIFF or other bitmap formats. The second stage is a deep neural network with multiple convolutional blocks to extract more effective features. We have conducted extensive experiments on three different image datasets. The experimental results demonstrate the superiority of our framework when compared with other state-of-the-art double JPEG compression detection methods either hand-crafted or learned using deep networks in the literature, especially in the two cases mentioned above. Furthermore, our proposed framework can detect triple and even multiple JPEG compressed images, which is scarce in the literature as far as we know.
Bidirectional Sign Language Translation

Communication is a tedious task for people who have hearing and speech impairment. Such people are assisted by a sign language translator for communicating with people. This however, is a slow process of translation, and many a times, it leads to loss of context in conversations, further leading to lack of involvement in social events. This research work proposes a portable and 24x7 available system with support for bidirectional translation i.e. from sign language to speech and speech to sign language. The portability of this system is assured with the use of a smartphone application, which is constantly available with the user, with no requirement of additional hardware. The mobile application will give normal speech output as audio and text and sign language output as a 3D animated video sequence, with the help of Unity3D. With the help of Machine Learning, the system will utilize a pre-trained model made with a combination of Convolution and Recurrent Neural Networks.
Instance-Level Relative Saliency Ranking with Graph Reasoning

Conventional salient object detection models cannot differentiate the importance of different salient objects. Recently, two works have been proposed to detect saliency ranking by assigning different degrees of saliency to different objects. However, one of these models cannot differentiate object instances and the other focuses more on sequential attention shift order inference. In this paper, we investigate a practical problem setting that requires simultaneously segment salient instances and infer their relative saliency rank order. We present a novel unified model as the first end-to-end solution, where an improved Mask R-CNN is first used to segment salient instances and a saliency ranking branch is then added to infer the relative saliency. For relative saliency ranking, we build a new graph reasoning module by combining four graphs to incorporate the instance interaction relation, local contrast, global contrast, and a high-level semantic prior, respectively. A novel loss function is also proposed to effectively train the saliency ranking branch. Besides, a new dataset and an evaluation metric are proposed for this task, aiming at pushing forward this field of research. Finally, experimental results demonstrate that our proposed model is more effective than previous methods. We also show an example of its practical usage on adaptive image retargeting.
csDetector: an open source tool for community smells detection

Community smells represent symptoms of sub-optimal organizational and social issues within software development communities that often lead to additional project costs and reduced software quality. Previous research identified a variety of community smells that are connected to sub-optimal patterns under different perspectives of organizational-social structures in the software development community. To detect community smells and understanding the characteristics of such organizational-social structures in a project, we propose csDetector, an open source tool that is able to automatically detect community smells within a project and provide relevant socio-technical metrics. csDetector uses a machine learning based detection approach that learns from various existing bad community development practices to provide automated support in detecting related community smells. We evaluate the effectiveness of csDetector on a benchmark of 143 open source projects from GitHub. Our results show that the csDetector tool can detect ten commonly occurring community smells in open software projects with an average F1 score of 84%. csDetector is publicly available, with a demo video, at: https://github.com/Nuri22/csDetector.
LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning

Deep metric learning has been effectively used to learn distance metrics for different visual tasks like image retrieval, clustering, etc. In order to aid the training process, existing methods either use a hard mining strategy to extract the most informative samples or seek to generate hard synthetics using an additional network. Such approaches face different challenges and can lead to biased embeddings in the former case, and (i) harder optimization (ii) slower training speed (iii) higher model complexity in the latter case. In order to overcome these challenges, we propose a novel approach that looks for optimal hard negatives (LoOp) in the embedding space, taking full advantage of each tuple by calculating the minimum distance between a pair of positives and a pair of negatives. Unlike mining-based methods, our approach considers the entire space between pairs of embeddings to calculate the optimal hard negatives. Extensive experiments combining our approach and representative metric learning losses reveal a significant boost in performance on three benchmark datasets <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Modeling learner-controlled mental model learning processes by a second-order adaptive network model

Learning knowledge or skills usually is considered to be based on the formation of an adequate internal mental model as a specific type of mental network. The learning process for such a mental model conceptualised as a mental network, is a form of (first-order) mental network adaptation. Such learning often integrates learning by observation and learning by instruction. For an effective learning process, an appropriate timing of these different elements is crucial. By controlling the timing of them, the mental network adaptation process becomes adaptive itself, which is called second-order mental network adaptation. In this paper, a second-order adaptive mental network model is proposed addressing this. The first-order adaptation process models the learning process of mental models and the second-order adaptation process controls the timing of the elements of this learning process. It is illustrated by a case study for the learner-controlled mental model learning in the context of driving a car. Here the learner is in control of the integration of learning by observation and learning by instruction.
A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods

Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts. In this paper, we present a comprehensive experimental evaluation and comparison with six state-of-the-art methods, from which we have re-implemented one of them. In addition, we investigate different textual and visual feature embeddings that cover different aspects of the content, as well as the recently introduced multimodal CLIP embeddings. Experimental results are presented for two different publicly available benchmark datasets of tweets and corresponding images. In contrast to the evaluation methodology of previous work, we introduce a reproducible and fair evaluation scheme to make results comparable. Finally, we conduct an error analysis to outline the limitations of the methods and possibilities for the future work.
Current Trends in Readmission Prediction: An Overview of Approaches

Hospital readmission shortly after discharge threatens the quality of patient care and leads to increased medical care costs. In the United States, hospitals with high readmission rates are subject to federal financial penalties. This concern calls for incentives for healthcare facilities to reduce their readmission rates by predicting patients who are at high risk of readmission. Conventional practices involve the use of rule-based assessment scores and traditional statistical methods, such as logistic regression, in developing risk prediction models. The recent advancements in machine learning driven by improved computing power and sophisticated algorithms have the potential to produce highly accurate predictions. However, the value of such models could be overrated. Meanwhile, the use of other flexible models that leverage simple algorithms offer great transparency in terms of feature interpretation, which is beneficial in clinical settings. This work presents an overview of the current trends in risk prediction models developed in the field of readmission. The various techniques adopted by researchers in recent years are described, and the topic of whether complex models outperform simple ones in readmission risk stratification is investigated.
Target-guided Emotion-aware Chat Machine

The consistency of a response to a given post at the semantic level and emotional level is essential for a dialogue system to deliver humanlike interactions. However, this challenge is not well addressed in the literature, since most of the approaches neglect the emotional information conveyed by a post while generating responses. This article addresses this problem and proposes a unified end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post and leveraging target information to generate more intelligent responses with appropriately expressed emotions. Extensive experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.
Comparison of domain adaptation techniques for white matter hyperintensity segmentation in brain MR images

Robust automated segmentation of white matter hyperintensities (WMHs) in different datasets (domains) is highly challenging due to differences in acquisition (scanner, sequence), population (WMH amount and location) and limited availability of manual segmentations to train supervised algorithms. In this work we explore various domain adaptation techniques such as transfer learning and domain adversarial learning methods, including domain adversarial neural networks and domain unlearning, to improve the generalisability of our recently proposed triplanar ensemble network, which is our baseline model. We used datasets with variations in intensity profile, lesion characteristics and acquired using different scanners. For the source domain, we considered a dataset consisting of data acquired from 3 different scanners, while the target domain consisted of 2 datasets. We evaluated the domain adaptation techniques on the target domain datasets, and additionally evaluated the performance on the source domain test dataset for the adversarial techniques. For transfer learning, we also studied various training options such as minimal number of unfrozen layers and subjects required for fine-tuning in the target domain. On comparing the performance of different techniques on the target dataset, domain adversarial training of neural network gave the best performance, making the technique promising for robust WMH segmentation.
SIDE: Center-based Stereo 3D Detector with Structure-aware Instance Depth Estimation

3D detection plays an indispensable role in environment perception. Due to the high cost of commonly used LiDAR sensor, stereo vision based 3D detection, as an economical yet effective setting, attracts more attention recently. For these approaches based on 2D images, accurate depth information is the key to achieve 3D detection, and most existing methods resort to a preliminary stage for depth estimation. They mainly focus on the global depth and neglect the property of depth information in this specific task, namely, sparsity and locality, where exactly accurate depth is only needed for these 3D bounding boxes. Motivated by this finding, we propose a stereo-image based anchor-free 3D detection method, called structure-aware stereo 3D detector (termed as SIDE), where we explore the instance-level depth information via constructing the cost volume from RoIs of each object. Due to the information sparsity of local cost volume, we further introduce match reweighting and structure-aware attention, to make the depth information more concentrated. Experiments conducted on the KITTI dataset show that our method achieves the state-of-the-art performance compared to existing methods without depth map supervision.
Improving sentiment analysis with learning concepts from concept, patterns lexicons and negations

The way of expressing sentiment (−ve/+ve) in the form of textual information depends on the way of thinking of human beings. Identifying aspect extraction and sentiment polarity from written texts is a crucial task. Mainly, a multi-level learning approach for aspect extraction from statistical methods, pattern-based methods, and rule-based methods. This work proposes the application of two probabilistic graphical Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) algorithms to generate latent topic terms as possible aspects. Then frequency-based and Concept lexicons are used to retrieve unigram to multi-word phrases with associated opinion words. Polarity shift is a significant issue that reverses the polarity of the aspects that affect the sentiment classification of the system. Therefore, to improve the performance of the machine learning classification algorithm in ABSA a hybrid approach comprising rule-based methods and a graph-theoretic model is applied to deal with the explicit and implicit polarity shift. The performance of the proposed method is measured using Naive Bayes, a machine learning classification algorithm on two datasets, SemEval 2014 Restaurant and SemEval 2014 Laptop dataset. Experimental result shows that the method for aspect extraction outperforms baseline methods by 86.32% and 82.64% for Restaurant, Laptop dataset, respectively. Similarly, for aspect-based sentiment classification, the accuracy and F1 measure on Restaurant domain 84.73%, 81.28% and 82.06% and 80.71% on the laptop domain.
Electromagnetic Side-Channel Analysis for IoT Forensics: Challenges, Framework, and Datasets

Electromagnetic (EM) side-channel radiation from Internet of Things (IoT) devices are shown to be effective at acquiring forensic insights during digital investigations.These EM radiation patterns can be analysed with the help of machine learning algorithms to detect internal behaviours of IoT devices, which can be relevant to an investigation.However, the real-world application of EM sidechannel analysis for digital forensic purposes is obstructed by the lack of suitable tools and the technical expertise among law-enforcement communities.Although certain frameworks, such as EMvidence, exist to cater this requirement, the sheer diversity of the IoT ecosystem makes it difficult to support a sufficiently large collection of devices that are commonly encountered in forensic investigations.The work presented in this paper makes multiple contributions towards addressing this problem.Initially, a detailed discussion on the challenges of applying EM side-channel analysis in practical digital forensic purposes is provided, where the practical difficulties are illustrated.Then, it was shown that the existing EM side-channel analysis frameworks, such as EMvidence, can be used to overcome the diversity of IoT devices in forensics by equipping them with extensible plug-ins targeting the internal system-on-chips (SoC) of each device type.These plug-ins are expected to incorporate trained machine learning models, which are capable of recognising patterns of specific IoT device SoCs.However, the development of such plug-ins requires sufficiently diverse EM datasets from IoT devices.Facilitating this requirement, this work presents a comprehensive EM side-channel dataset representing a diverse collection of popular IoT devices and smartphones.The presented dataset is used to demonstrate the potential usage of machine learning models to recognise device behaviour.
Deep residual learning in modulation recognition of radar signals using higher-order spectral distribution

Automatically recognizing intra-pulse modulation of radar signals is a significant survival technique in electronic intelligence systems. To avoid the dependence on feature selection and realize the intelligent intra-pulse modulation recognition of various radar signals under low signal-to-noise ratios (SNRs), this paper develops a novel intra-pulse modulation recognition method based on the high-order spectrums of radar signals. Automatic soft thresholding is implemented in the deep residual network to adaptively eliminate redundant information in the process of feature learning and improve the learning effect of valuable features in distribution images of corresponding third-order spectrums. The extensive simulations compared with the other four methods further reveal the excellent classification performance of the proposed method. The proposed approach still achieves an overall probability of successful recognition of 93.5% for eight kinds of modulation signals, even when the SNR is just −8 dB. Outstanding performance proves the superiority and robustness of the proposed method.
Research on Pseudorandom Number Generator Based on Several New Types of Piecewise Chaotic Maps

This paper proposes three types of one-dimensional piecewise chaotic maps and two types of symmetrical piecewise chaotic maps and presents five theorems. Furthermore, some examples that satisfy the theorems are constructed, and an analysis and model of the dynamic properties are discussed. The construction methods proposed in this paper have a certain generality and provide a theoretical basis for constructing a new discrete chaotic system. In addition, this paper designs a pseudorandom number generator based on piecewise chaotic map and studies its application in cryptography. Performance evaluation shows that the generator can generate high quality random sequences efficiently.
A Full-Featured FPGA-Based Pipelined Architecture for SIFT Extraction

Image feature detection is a key task in computer vision. Scale Invariant Feature Transform (SIFT) is a prevalent and well known algorithm for robust feature detection. However, it is computationally demanding and software implementations are not applicable for real-time performance. In this paper, a versatile and pipelined hardware implementation is proposed, that is capable of computing keypoints and rotation invariant descriptors on-chip. All computations are performed in single precision floating-point format which makes it possible to implement the original algorithm with little alteration. Various rotation resolutions and filter kernel sizes are supported for images of any resolution up to ultra-high definition. For full high definition images, 84 fps can be processed. Ultra high definition images can be processed at 21 fps.
Exquisitor at the Lifelog Search Challenge 2021: Relationships Between Semantic Classifiers

Exquisitor is a scalable media exploration system based on interactive learning. To satisfy a user's information need, the system asks the user for feedback on media items and uses that feedback to interactively construct a classifier, that is in turn used to identify the next potentially relevant set of media items. To facilitate effective exploration of a collection, the system offers filters to narrow the scope of exploration, search functionality for finding good examples for the classifier, and support for timeline browsing of videos or image sequences. For this year's Lifelog Search Challenge, we have enhanced Exquisitor to better support tasks with a temporal component, by adding features that allow the user to build multiple classifiers and merge the classifier results, using both traditional set operators and advanced temporal operators.