Reviewerly/exHarmony huggingface dataset:
Introducing Semantic Information for Numerical Attribute Prediction over Knowledge Graphs

Knowledge graph (KG) completion has been long studied on link prediction task to infer missing relations, while literals are paid less attention due to the non-discrete and rich-semantic challenges. Numerical attributes such as height, age and birthday are different from other literals that they can be calculated and estimated, thus have huge potential to be predicted and play important roles in a series of tasks. However, only a few researches have made preliminary attempts to predict numerical attributes on KGs with the help of the structural information or the development of embedding techniques. In this paper, we re-examine the numerical attribute prediction task over KGs, and introduce several novel methods to explore and utilize the rich semantic knowledge of language models (LMs) for this task. An effective combination strategy is also proposed to take full advantage of both structural and semantic information. Extensive experiments are conducted to show the great effectiveness of both the semantic methods and the combination strategy.
PSCVR: Physiological Sensing in Collaborative Virtual Reality

In a collaborative environment where more than a single user need to complete a task together, being able to understand each other's emotional status is essential for a successful outcome. This is especially evident in remote collaboration tasks where the physical distance may create a lack of understanding, empathy, and comprehension be-tween the partners. Typical remote collaboration systems share less information than that can be communicated in a physical interaction, which makes supporting awareness very challenging. Our proposed visual representation allows users to infer emotional patterns from physiological data, which could impact their communication style towards a more forceful or inactive and calm association. We investigate the potential effects of the proposed visual rep-resentation to support empathetic communication during remote collaboration, as well as the design guidelines for building such systems.
Fast and Accurate Evaluation of Collaborative Filtering Recommendation Algorithms

Collaborative filtering are recommender systems algorithms that provide personalized recommendations to users in various online environments such as movies, music, books, jokes and others. There are many such recommendation algorithms and, regarding experimental evaluations to find which algorithm performs better a lengthy process needs to take place and the time required depends on the size of the dataset and the evaluation metrics used. In this paper we present a novel method that is based on a series of steps that include random subset selections, ensemble learning and the use of well-known evaluation metrics Mean Absolute Error and Precision to identify, in a fast and accurate way, which algorithm performs the best for a given dataset. The proposed method has been experimentally evaluated using two publicly available datasets with the experimental results showing that the time required for the evaluation is significantly reduced, while the results are accurate when compared to a full evaluation cycle.
Data-Out Instruction-In (DOIN!): Leveraging Inclusive Caches to Attack Speculative Delay Schemes

Although the cache has been a known side-channel for years, it has gained renewed notoriety with the introduction of speculative side-channel attacks such as Spectre, which were able to use caches to not just observe a victim, but to leak secrets. Because the cache continues to be one of the most exploitable side channels, it is often the primary target to safeguard in secure speculative execution schemes. One of the simpler secure speculation approaches is to delay speculative accesses, whose effect can be observed, until they become non-speculative. Delay-on-Miss, for example, delays all observable speculative loads, i.e., loads that miss in the cache, and preserves the majority of the performance of the baseline (unsafe speculation) by executing speculative loads that hit in the cache, which were thought to be unobservable. However, previous works have failed to consider how instruction fetching can eject cache lines from the shared, lower-level caches, and thus from higher cache levels due to inclusivity. In this work, we show how cache conflicts between instruction fetch and data accesses can extend previous attacks, and we present the following new insights: 1) It is possible to use lowerlevel caches to perform Prime+Probe through conflicts resulting from instruction fetching. This is an extension to previous Prime+Probe attacks that potentially avoids other developed mitigation strategies. 2) Data-instruction conflicts can be used to perform a Spectre attack that breaks Delay-on-Miss. After acquiring a secret, secret-dependent instruction fetching can cause cache conflicts that result in evictions in the L1D cache, creating observable timing differences. Essentially, it is possible to leak a secret bit-by-bit through the cache, despite Delay-on-Miss defending against caches. We call our new attack Data-Out Instruction-In, DOIN!, and demonstrate it on a real commercial core, the AMD Ryzen 9. We demonstrate how DOIN! interacts with Delay-on-Miss and perform an analysis of noise and bandwidth. Furthermore, we propose a simple defense extension for Delay-on-Miss to maintain its security guarantees, at the cost of negligible performance degradation while executing the SPEC CPU2006 workloads.
Explainable and Generalizable Blind Image Quality Assessment via Semantic Attribute Reasoning

Blind image quality assessment (BIQA) that can directly evaluate image quality without perfect-quality reference has been a long-standing research topic. Although the existing BIQA models have achieved very encouraging performance, the lack of explainability and generalization ability limits their real-world applications to a great extent. People usually assess image quality according to semantic attributes, e.g., brightness, color, contrast, noise and sharpness. Furthermore, judgment on image quality is also impacted by the scene presented in the image. Therefore, the inherent relationship between semantic attributes and scenes is crucial for image quality assessment, which has rarely been explored yet. With this motivation, this paper presents a Semantic Attribute Reasoning based image QUality Evaluator (SARQUE). Specifically, we propose a two-stream network to predict semantic attributes and scene categories from distorted images. To investigate the inherent relationship between the semantic attributes and scene category, a semantic reasoning module is further proposed based on the graph convolution network (GCN), producing the final quality score. Extensive experiments conducted on five in-the-wild image quality databases demonstrate the superiority of the proposed SARQUE model over the state-of-the-arts. Furthermore, the proposed model features better explainability and generalization ability due to the use of semantic attributes.
A Deeper CNN Approach For Detection Of Collapsed Buildings In Drone Images

Building collapses lead to high mortality rate and other damages. UAVs attached with optical cameras are widely available and can be used for collapsed building detection as they can cover a larger area within a given time with minimal cost. This paper is a systematic study on utilization of deep learning based approaches for collapsed building detection on images collected by UAVs. This study aims to identify suitable deep learning models with focus on optimization for real world application scenarios. Five different pre-trained Convolutional Neural Networks (CNN) are trained using transfer learning on our custom data set to assess the performance. The real novelty in this study pertains to suitably enhancing the architecture of these models by increasing the depth and width of the neural network models on top of feature extraction and fine tuning approaches, without any substantial increase in the additional computational tasks. Another prominent contribution is the construction of a new drone-based building image data set with more emphasis on real world correlation between collapsed and normal buildings from disaster hit areas. The enhanced CNN architectures of the five models applied to this rich data set resulted in an average 6.67% increase in accuracy as compared to the general fine tuning approaches. Through deeper fine tuning approach, model training time was also reduced by 6 seconds in average with moderate increase in trained model size.
A Low-Complexity Approach to Rate-Distortion Optimized Variable Bit-Rate Compression for Split DNN Computing

Split computing has emerged as a recent paradigm for implementation of DNN-based AI workloads, wherein a DNN model is split into two parts, one of which is executed on a mobile/client device and the other on an edge-server (or cloud). Data compression is applied to the intermediate tensor from the DNN that needs to be transmitted, addressing the challenge of optimizing the rate-accuracy-complexity trade-off. Existing split-computing approaches adopt ML-based data compression, but require that the parameters of either the entire DNN model, or a significant portion of it, be retrained for different compression levels. This incurs a high computational and storage burden: training a full DNN model from scratch is computationally demanding, maintaining multiple copies of the DNN parameters increases storage requirements, and switching the full set of weights during inference increases memory bandwidth. In this paper, we present an approach that addresses all these challenges. It involves the systematic design and training of bottleneck units - simple, low-cost neural networks - that can be inserted at the point of split. Our approach is remarkably lightweight, both during training and inference, highly effective and achieves excellent rate-distortion performance at a small fraction of the compute and storage overhead compared to existing methods.
Machine Learning Based Personalized Movie Research and Implementation of Recommendation System

With the development of the Internet industry, the information age presents a trend of “information overload”, and people's efficiency in extracting effective information is getting lower and lower. In order to relieve people's browsing pressure, this paper implements a collaborative filtering algorithm based on machine learning for the movie recommendation, citing the principle of personalized recommendation system proposed by Robert Armstrong and others in the United States in 1995. First, the rating data is preprocessed and visualized in consideration of the user's real behavior. Then implement the algorithm mentioned above, and use the test indicators to measure the performance of the recommender system and optimize the system parameters. Finally, using software engineering and Java front-end knowledge based on Spring+SpringMVC+Mybaits (SSM) to conduct demand analysis, functional analysis, non-functional analysis and establish a database. At last, use java database connectivity (JDBC) to link the database Mysql, and finally realized a movie recommender system with basic functions.
Pay attention to emoji: Feature Fusion Network with EmoGraph2vec Model for Sentiment Analysis

With the explosive growth of social media, opinionated postings with emojis have increased explosively. Many emojis are used to express emotions, attitudes, and opinions. Emoji representation learning can be helpful to improve the performance of emoji-related natural language processing tasks, especially in text sentiment analysis. However, most studies have only utilized the fixed descriptions provided by the Unicode Consortium without consideration of actual usage scenarios. As for the sentiment analysis task, many researchers ignore the emotional impact of the interaction between text and emojis. It results that the emotional semantics of emojis cannot be fully explored. In this work, we propose a method called EmoGraph2vec to learn emoji representations by constructing a co-occurrence graph network from social data and enriching the semantic information based on an external knowledge base EmojiNet to embed emoji nodes. Based on EmoGraph2vec model, we design a novel neural network to incorporate text and emoji information into sentiment analysis, which uses a hybrid-attention module combined with TextCNN-based classifier to improve performance. Experimental results show that the proposed model can outperform several baselines for sentiment analysis on benchmark datasets. Additionally, we conduct a series of ablation and comparison experiments to investigate the effectiveness and interpretability of our model.
Inference of Multi-Class STL Specifications for Multi-Label Human-Robot Encounters

This paper is interested in formalizing human trajectories in human-robot encounters. Inspired by robot navigation tasks in human-crowded environments, we consider the case where a human and a robot walk towards each other, and where humans have to avoid colliding with the incoming robot. Further, humans may describe different be-haviors, ranging from being in a hurry/minimizing completion time to maximizing safety. We propose a decision tree-based algorithm to extract STL formulae from multi-label data. Our inference algorithm learns STL specifications from data containing multiple classes, where instances can be labelled by one or many classes. We base our evaluation on a dataset of trajectories collected through an online study reproducing human-robot encounters.
TSDet: End-to-End Method with Transformer for SAR Ship Detection

Synthetic Aperture Radar (SAR) ship object detection has important applications in maritime monitoring. Currently, CNN-based detectors are hardly perfect for multi-scale SAR ship detection by scattered noise and complex land and sea background interference. Therefore, we propose a new end-to-end method with Transformer for SAR ship detection (named TSDet). TSDet adopts an anchor-free design that can directly predict object bounding boxes and classes of multi-scale ships. Specifically, we propose Perceptual Enhancement Transformer (PET) at first, as the network structure of TSDet, which consists of the Feature Perception Enhancement (FPE) module based on self-attention and the Task Perception Enhancement (TPE) module based on cross-attention. PET can effectively suppress background noise in SAR images and enhance the salient features of ships. Second, to solve the sparse features of SAR ships, we propose a new effective sparse attention alternative to the original attention module in PET networks. It can quickly focus on important information in global features and accelerate network convergence. Finally, on the SAR-Ship dataset, our TSDet achieves the best 93.77 % AP <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</inf> , outperforming competitive benchmarks.
Membership Inference Attacks Against Deep Learning Models via Logits Distribution

Deep Learning(DL) techniques have gained significant importance in the recent past due to their vast applications. However, DL is still prone to several attacks, such as the Membership Inference Attack (MIA), based on the memorability of training data. MIA aims at determining the presence of specific data in the training dataset of the model with substitute model of similar structure to the objective model. As MIA relies on the substitute model, they can be mitigated if the substitute model is not clear about the network structure of the objective model. To solve the challenge of shadow-model construction, this work presents L-Leaks, a member inference attack based on Logits. L-Leaks allow an adversary to use the substitute model's information to predict the presence of membership if the shadow and objective model are similar enough. Here, the substitute model is built by learning the logits of the objective model, hence making it similar enough. This results in the substitute model having sufficient confidence in the member samples of the objective model. The evaluation of the attack's success shows that the proposed technique can execute the attack more accurately than existing techniques. It also shows that the proposed MIA is significantly robust under different network models and datasets.
Layered Depth Refinement with Mask Guidance

Depth maps are used in a wide range of applications from 3D rendering to 2D image effects such as Bokeh. However, those predicted by single image depth estimation (SIDE) models often fail to capture isolated holes in objects and/or have inaccurate boundary regions. Meanwhile, high-quality masks are much easier to obtain, using commercial auto-masking tools or off-the-shelf methods of segmentation and matting or even by manual editing. Hence, in this paper, we formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models. Our framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask. As datasets with both depth and mask annotations are scarce, we propose a self-supervised learning scheme that uses arbitrary masks and RGB-D datasets. We empirically show that our method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions. We further analyze our model with an ablation study and demonstrate results on real applications. More information can be found on our project page. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://sooyekim.github.io/MaskDepth/
Interpretation of Expressions through Hand Signs Using Deep Learning Techniques

It is a challenging task to interpret sign language automatically, as it comprises high-level vision features to accurately understand and interpret the meaning of the signer or vice versa. In the current study, we automatically distinguish hand signs and classify seven basic gestures representing symbolic emotions or expressions like happy, sad, neutral, disgust, scared, anger, and surprise. Convolutional Neural Network is a famous method for classifications using vision-based deep learning; here in the current study, proposed transfer learning using a well-known architecture of VGG16 to speed up the convergence and improve accuracy by using pre-trained weights. We obtained a high accuracy of 99.98% of the proposed architecture with a minimal and low-quality data set of 455 images collected by 65 individuals for seven hand gesture classes. Further, compared the performance of VGG16 architecture with two different optimizers, SGD, and Adam, along with some more architectures of Alex Net, LeNet05, and ResNet50.
A Secure Framework for Internet of Medical Things Security Based System Using Lightweight Cryptography Enabled Blockchain

The Internet of Medical Things (IoMT) is a growing paradigm that offers several efficient and productive solutions for the treatment of various ailments for both patients and medical professionals. The IoMT has many advantages, but security remains a problem that must be overcome. The inexperienced IoMT users’ lack of security and privacy consciousness and the possibility of multiple middleman attacks for getting the healthcare information, seriously puts the use of IoMT in jeopardy. Therefore, this paper proposes a lightweight cryptography enabled with blockchain for enhancing IoMT-based security and privacy. The study utilizes lightweight cryptography to securely uploading the data to the cloud database for privacy preservation, and the Blockchain technology is use to securely store the data in the cloud server. The experimental results of the proposed model revealed a better result with compare with prevailing methods. The proposed system achieves an accuracy of 98% of security level.
Research on Neural Graph Collaborative Filtering Recommendation Model Fused With Item Temporal Sequence Relationships

Graph neural network is blossoming recently,and it can explicitly express user-item high-order connectivity information,so it can significantly improve the recommendation performance when it applied to recommender system. However, the existing methods usually assume that the user’s interest is invariant,and there is insufficient explore to charactertize the user’s dynamic interest changes through the temporal sequences dependencies of items. In this paper, we propose a research on graph neural network collaborative recommendation model fused item temporal sequence relation, that is, a top-N hybrid recommendation model that fuses user-item interaction information and item temporal sequences dependencies. It divides the item temporal sequences into several groups of subsequences through the sliding window, constructs the item temporal sequences dependency graph, aggregates the characteristics of item temporal sequences information, and deeply depict the dynamic changes of users’ interests, and uses the bipartite graph neural network to map the high-dimensional information of user-item and item-item to the low-dimensional space. The hybrid embedding of user-item historical interaction information and item temporal sequences dependency information is realized, and the expression of user-item interaction sequence information is enhanced. Finally, through the user-item interaction graph and item sequence dependency graph, constructed a graph neural collaborative filtering recommendation framework including embedding layer,aggregation layer,propagation layer and prediction layer. Experiments demonstrate that the model performance has been significantly improved on datasets such as LastFM, Ciao and Douban.
INS-Conv: Incremental Sparse Convolution for Online 3D Segmentation

We propose INS-Conv, an INcremental Sparse Convolutional network which enables online accurate 3D semantic and instance segmentation. Benefiting from the incremental nature of RGB-D reconstruction, we only need to update the residuals between the reconstructed scenes of consecutive frames, which are usually sparse. For layer design, we define novel residual propagation rules for sparse convolution operations, achieving close approximation to standard sparse convolution. For network architecture, an uncertainty term is proposed to adaptively select which residual to update, further improving the inference accuracy and efficiency. Based on INS-Conv, an online joint 3D semantic and instance segmentation pipeline is proposed, reaching an inference speed of 15 FPS on GPU and 10 FPS on CPU. Experiments on ScanNetv2 and SceneNN datasets show that the accuracy of our method surpasses previous online methods by a large margin, and is on par with state-of-the-art offline methods. A live demo on portable devices further shows the superior performance of INS-Conv.