Reviewerly/exHarmony huggingface dataset:
Heterogeneous information networks

In 2011, we proposed PathSim to systematically define and compute similarity between nodes in a heterogeneous information network (HIN), where nodes and links are from different types. In the PathSim paper, we for the first time introduced HIN with general network schema and proposed the concept of meta-paths to systematically define new relation types between nodes. In this paper, we summarize the impact of PathSim paper in both academia and industry. We start from the algorithms that are based on meta-path-based feature engineering, then move on to the recent development in heterogeneous network representation learning, including both shallow network embedding and heterogeneous graph neural networks. In the end, we make the connection between knowledge graphs and HINs and discuss the implication of meta-paths in the symbolic reasoning scenario. Finally, we point out several future directions.
Practical and Scalable Desktop-Based High-Quality Facial Capture

We present a novel desktop-based system for high-quality facial capture including geometry and facial appearance. The proposed acquisition system is highly practical and scalable, consisting purely of commodity components. The setup consists of a set of displays for controlled illumination for reflectance capture, in conjunction with multiview acquisition of facial geometry. We additionally present a novel set of modulated binary illumination patterns for efficient acquisition of reflectance and photometric normals using our setup, with diffuse-specular separation. We demonstrate high-quality results with two different variants of the capture setup – one entirely consisting of portable mobile devices targeting static facial capture, and the other consisting of desktop LCD displays targeting both static and dynamic facial capture.
Transformer-Based Feature Compensation and Aggregation for DeepFake Detection

Deepfake detection has attracted increasing attention in recent years. In this paper, we propose a transformer-based framework with feature compensation and aggregation (Trans-FCA) to extract rich forgery cues for deepfake detection. To compensate local features for transformers, we propose a Locality Compensation Block (LCB) containing a Global-Local Cross-Attention (GLCA) to attentively fuse global transformer features and local convolutional features. To aggregate features of all layers for capturing comprehensive and various fake flaws, we propose Multi-head Clustering Projection (MCP) and Frequency-guided Fusion Module (FFM), where the MCP attentively reduces redundant features into a few concentrated clusters, and the FFM interacts all clustered features under the guidance of frequency cues. In Trans-FCA, besides global cues captured by transformer architecture, local details and rich forgery defects are also captured using the proposed fetaure compensation and aggregation. Extensive experiments show our method outperforms the state-of-the-art methods on both intra-dataset and cross-dataset testings (with AUCs of 99.85% on FaceForensics++ and 78.57% on Celeb-DF), which clearly demonstrates the superiority of our Trans-FCA for deepfake detection.
ARFA: Adaptive Reception Field Aggregation for 3-D Detection From LiDAR Point Cloud

Submanifold convolution is widely used in 3-D detection. However, it brings different receptive fields to voxels due to the nonuniform distribution in Light Detection and Ranging (LiDAR) point clouds, resulting in degradation of the feature extraction ability for distant voxels and the performance of detectors. We propose a solution, adaptive receptive field aggregation (ARFA) network, an end-to-end two-stage LiDAR 3-D object detection architecture. ARFA searches the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> nearest neighbors (KNNs) to adaptively adjust the receptive field of sparse voxels, followed by a self-attention aggregation (SA) module with density feature embedding (DE) to aggregate the semantic information in the receptive field. In order to further strengthen the detection performance for small objects, we also propose an upsampling bird’s eyes view (U-BEV) backbone and a Intersection over Union (IoU)-aware head to enhance the quality of the proposals and rectify the confidence of the predicted bounding boxes. ARFA outperforms the state-of-the-art methods on the Waymo Open dataset and achieves competitive results on the popular KITTI dataset.
Switch and Refine: A Long-Term Tracking and Segmentation Framework

In long-term video object tracking (VOT) tasks, most long-term trackers are modified from short-term trackers, which contain more and more machine learning modules to improve their performance. However, we empirically find that more modules do not necessarily lead to better results. In this paper, we make the long-term tracking framework simple by carefully selecting the cutting-edge trackers. Specifically, we propose a new long-term VOT framework that combines the benefits of two mainstream short-term tracking pipelines, i.e., the discriminative online tracker and the one-shot Siamese tracker, with a global re-detector awakened when the target is lost. Such a framework fully exploits existing advanced works from three complementary perspectives. Experimental results show that by exploiting the capabilities of existing methods instead of designing new neural networks, we can still achieve remarkable results on seven long-term VOT datasets. By introducing a continuous adjustable speed control parameter, our tracker reaches 20+FPS with only a small performance loss. The refine module not only improves the bounding box estimations but also outputs segmentation masks, so that our framework can handle the video object segmentation (VOS) tasks by using only VOT trackers. We obtain a trade-off between time and accuracy on two representative VOS datasets by only using bounding boxes as the initial input.
Use of Artificial Intelligence Systems for Determining the Career Guidance of Future University Student

This study is devoted to solving the problem of determining the career guidance of future university student using artificial intelligence systems. It is proposed Fully Connected Feed-Forward Neural Network (FNN) architecture and performed empirical modeling to create a Data Set. Model of artificial intelligence system allows evaluating the processes in an FNN during the execution of multi-label classification of professional areas. A method has been developed for the training process of a machine learning model, which reflects the internal connections between the components of an artificial intelligence system that allow it to “learn” from training data. To train the neural network, a data set of 29 input parameters and 23 output parameters was used; the amount of data in the set is 936 data lines. The software product was developed by Python and uses Keras, Numpy and Pandas libraries. Results of this research can be used to further improve the knowledge and skills necessary for successful professional realization.
Card-Based Zero-Knowledge Proof for the Nearest Neighbor Property: Zero-Knowledge Proof of ABC End View

This paper shows a zero-knowledge proof protocol of a solution to ABC end view puzzle using physical cards. Card-based cryptographic protocols are proposed to execute a secure multi-party calculation using physical cards instead of computers. This paper shows a card-based zero-knowledge proof of the ABC end view puzzle. The puzzle needs a new technique to prove the nearest neighbor from an end. We show a new zero-knowledge proof protocol to securely calculate the nearest neighbor using physical cards.
A Tool For Debugging Quantum Circuits

As the scale of quantum programs grows to match that of classical software, the nascent field of quantum software engineering must mature and tools such as debuggers will become increasingly important. However, developing a quantum debugger is challenging due to the nature of a quantum computer; sneaking a peek at the value of a quantum state will cause either partial or complete collapse of the superposition and may destroy the necessary entanglement. As a first step to developing a full quantum circuit debugger, we have designed and implemented a quantum circuit debugging tool. The tool allows the user to divide the circuit vertically or horizontally into smaller chunks known as slices, and manage their simulation or execution for either interactive debugging or automated testing. The tool also enables developers to track gates within the overall circuit and each chunk to understand their behavior better. Feedback on usefulness and usability from early users shows that using the tool to slice and test their circuits has helped make the debugging process more time-efficient for them.
Markov Decision Process for Modeling Social Engineering Attacks and Finding Optimal Attack Strategies

It is important to comprehend the attacker’s behavior and capacity in order to build a stronger fortress and thus be able to protect valuable assets more effectively. Prior to launching technical and physical attacks, an attacker may enter the reconnaissance stage and gather sensitive information. To collect such valuable data, one of the most effective approaches is through conducting social engineering attacks, borrowing techniques from deception theory. As a result, it is of utmost importance to understand when an attacker behaves truthfully and when the attacker opts to be deceitful. This paper models attacker’s states using Markov Decision Process (MDP) and studies the attacker’s decision for launching deception attacks in terms of cooperation and deception costs. The study is performed through Markov Decision Process (MDP) modeling, where the states of attackers are modeled along with the permissible actions that can be taken. We found that the optimal policy regarding being deceitful or truthful depends on the cost associated with deception and how much the attacker can afford to take the risk of launching deception attacks. More specifically, we observed that when the cost of cooperation is low (e.g., 10%), by taking MDP optimal policy, the attacker cooperates with the victim as much as possible in order to gain their trust; whereas, when the cost of cooperation is high (e.g., 50%), the attacker takes deceptive action earlier in order to minimize the cost of interactions while maximizing the impact of the attack. We report four case studies and simulations through which we demonstrate the trade-off between cooperative and deceptive actions in accordance with their costs to attackers.
TIGRIS: An Informed Sampling-based Algorithm for Informative Path Planning

Informative path planning is an important and challenging problem in robotics that remains to be solved in a manner that allows for wide-spread implementation and real-world practical adoption. Among various reasons for this, one is the lack of approaches that allow for informative path planning in high-dimensional spaces and non-trivial sensor constraints. In this work we present a sampling-based approach that allows us to tackle the challenges of large and high-dimensional search spaces. This is done by performing informed sampling in the high-dimensional continuous space and incorporating potential information gain along edges in the reward estimation. This method rapidly generates a global path that maximizes information gain for the given path budget constraints. We discuss the details of our implementation for an example use case of searching for multiple objects of interest in a large search space using a fixed-wing UAV with a forward-facing camera. We compare our approach to a sampling-based planner baseline and demonstrate how our contributions allow our approach to consistently out-perform the baseline by 18.0%. With this we thus present a practical and generalizable informative path planning framework that can be used for very large environments, limited budgets, and high dimensional search spaces, such as robots with motion constraints or high-dimensional configuration spaces.
Safety Certification for Stochastic Systems via Neural Barrier Functions

Providing non-trivial certificates of safety for non-linear stochastic systems is an important open problem. One promising solution to address this problem is the use of barrier functions. Barrier functions are functions whose composition with the system forms a Martingale and enable the computation of the probability that the system stays within a safe set over a finite time horizon. However, existing approaches to find barrier functions generally restrict the search to a small class of functions, often leading to conservatism. To address this problem, in this letter, we parameterize barrier functions as neural networks and show that bound propagation techniques and linear programming can be successfully employed to find Neural Barrier Functions. Further, we develop a branch-and-bound scheme based on linear relaxations that improves the scalability of the proposed framework. On several case studies we show that our approach scales to neural networks of hundreds of neurons and multiple hidden layers and often produces certificates of safety that are tighter than state-of-the-art methods.
Joint Luma and Chroma Multi-Scale CNN In-loop Filter for Versatile Video Coding

In this paper, a model selection based multi-scale convolutional neural network (CNN) model for in-loop filtering is proposed for Versatile Video Coding (VVC). We propose a novel network to jointly filter the luminance component and chrominance components simultaneously in the coding loop to improve the quality of the reconstructed frames. The proposed model contains two main branches with different scales as well as the global identity connection. Each branch contains several residual blocks. We fuse the residual features of the two branches by using a convolutional block attention module (CBAM) [1] at the end of the model. In addition, we adopt a model selection strategy that can select the best coding tree unit (CTU) level model in terms of R-D cost at the encoder. Compared with VTM-14.0 our method achieves 5.17%, 11.05% and 10.89% BD-rate reduction under all intra (AI) configuration, and 6.56%, 13.14%, 11.98% BD-rate reduction under random access (RA) configuration. Moreover, under RA configuration, the proposed model selection strategy brings extra 0.34% BD-rate reduction in the luminance component.
Blind Image Decomposition

We propose and study a novel task named Blind Image Decomposition (BID), which requires separating a superimposed image into constituent underlying images in a blind setting, that is, both the source components involved in mixing as well as the mixing mechanism are unknown. For example, rain may consist of multiple components, such as rain streaks, raindrops, snow, and haze. Rainy images can be treated as an arbitrary combination of these components, some of them or all of them. How to decompose superimposed images, like rainy images, into distinct source components is a crucial step toward real-world vision systems. To facilitate research on this new task, we construct multiple benchmark datasets, including mixed image decomposition across multiple domains, real-scenario deraining, and joint shadow/reflection/watermark removal. Moreover, we propose a simple yet general Blind Image Decomposition Network (BIDeN) to serve as a strong baseline for future work. Experimental results demonstrate the tenability of our benchmarks and the effectiveness of BIDeN. Codes and datasets are available at GitHub .
W2N: Switching from Weak Supervision to Noisy Supervision for Object Detection

Weakly-supervised object detection (WSOD) aims to train an object detector only requiring the image-level annotations. Recently, some works have managed to select the accurate boxes generated from a well-trained WSOD network to supervise a semi-supervised detection framework for better performance. However, these approaches simply divide the training set into labeled and unlabeled sets according to the image-level criteria, such that sufficient mislabeled or wrongly localized box predictions are chosen as pseudo ground-truths, resulting in a sub-optimal solution of detection performance. To overcome this issue, we propose a novel WSOD framework with a new paradigm that switches from weak supervision to noisy supervision (W2N). Generally, with given pseudo ground-truths generated from the well-trained WSOD network, we propose a two-module iterative training algorithm to refine pseudo labels and supervise better object detector progressively. In the localization adaptation module, we propose a regularization loss to reduce the proportion of discriminative parts in original pseudo ground-truths, obtaining better pseudo ground-truths for further training. In the semi-supervised module, we propose a two tasks instance-level split method to select high-quality labels for training a semi-supervised detector. Experimental results on different benchmarks verify the effectiveness of W2N, and our W2N outperforms all existing pure WSOD methods and transfer learning methods. Our code is publicly available at https://github.com/1170300714/w2n_wsod .
Analysis on Blockchain Consensus Mechanism Based on Proof of Work and Proof of Stake

In the white book of Bitcion, Satoshi Nakamoto described a bitcoin system that can realize point-to-point online payment without a third-party organization. After supporting this magical application scenario and subverting the traditional centralized system, the blockchain technology has attracted worldwide attention, triggered a research upsurge of blockchain consensus algorithm, and produced a large number of innovative applications. Although various consensus algorithms continue to evolve with the iteration of blockchain products and applications, Proof of Work (POW) and Proof of Skake (POS) algorithms are still the core of consensus algorithms. This paper discussestwo algorithms of POW and POS in blockchain consensus mechanism, and analyzes the advantages and the existing problems of the two consensus mechanisms. Since consensus mechanism is the main focus of blockchain technology and has many influencing factors, this paper discusses the current problems and some improved ideas, and selectes some typical algorithms for a more systematic introduction. In addition, some important issues related to safety and performance are also discussed. This paper provides the researchers a great reference on blockchain consensus mechanism.
Adaptive Learning Support System Based on Automatic Recommendation of Personalized Review Materials

In this study, we propose an integrated system to support learners' reviews.In the proposed system, the review dashboard is used to recommend review contents that are adaptive to the individual learner's level of understanding and to present other information that is useful for review.The pages of the digital learning materials that are estimated to be insufficiently understood by each learner and the webpages related to those pages are recommended.As a method for estimating such pages, we consider extracting the pages related to the questions that were answered incorrectly.We examined the accuracy of matching each question with the pages of the learning materials.We also conducted an experiment to verify the usefulness of the system and its effect on learning using a review dashboard.In the experiment, the evaluation of the review dashboard indicated that at least half of the participants found it useful for most types of feedback.In addition, the rate of change in quiz scores was significantly higher in the group using the review dashboard, which indicates that using the review dashboard has the effect of improving learning.
Classical to Quantum Software Migration Journey Begins: A Conceptual Readiness Model

With recent advances in the development of more powerful quantum computers, the research area of quantum software engineering is emerging. Quantum software plays a critical role in exploiting the full potential of quantum computing systems. As a result, it has been drawing increasing attention recently to provide concepts, principles, and guidelines to address the ongoing challenges of quantum software development. The importance of the topic motivated us to voice out a call for action to develop a readiness model that will help an organization assess its capability of migration from classic software engineering to quantum software engineering. The proposed model will be based on the existing multivocal literature, industrial empirical study, understanding of the process areas, challenging factors and enablers that could impact the quantum software engineering process. We believe that the proposed model will provide a roadmap for software development organizations to measure their readiness concerning to transformation from classic to quantum software engineering by suggesting best practices and highlighting important process areas, challenges, and enablers.
Multi-Task Text Classification using Graph Convolutional Networks for Large-Scale Low Resource Language

Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on resource-rich languages like English. Applying GCN for multi-task text classification is an unexplored area. Moreover, training a GCN or adopting an English GCN for Indian languages is often limited by data availability, rich morphological variation, syntax, and semantic differences. In this paper, we study the use of GCN for the Telugu language in single and multi-task settings for four natural language processing (NLP) tasks, viz. sentiment analysis (SA), emotion identification (EI), hate-speech (HS), and sarcasm detection (SAR). In order to evaluate the performance of GCN with one of the Indian languages, Telugu, we analyze the GCN based models with extensive experiments on four downstream tasks. In addition, we created an annotated Telugu dataset, TEL-NLP, for the four NLP tasks. Further, we propose a supervised graph reconstruction method, Multi-Task Text GCN (MT- Text GCN) on the Telugu that leverages to simultaneously (i) learn the low-dimensional word and sentence graph embeddings from word-sentence graph reconstruction using graph autoencoder (GAE) and (ii) perform multi-task text classification using these latent sentence graph embeddings. We argue that our proposed MT- Text GCN achieves significant improvements on TEL-NLP over existing Telugu pretrained word embeddings [1], multilingual pretrained Transformer models: mBERT [2], and XLM-R [3]. On TEL-NLP, we achieve a high Fl-score for four NLP tasks: SA (0.84), EI (0.55), HS (0.83) and SAR (0.66). Finally, we show our model's quantitative and qualitative analysis on the four NLP tasks in Telugu. We open-source our TEL-NLP dataset, pretrained models, and code <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> https://github.com/scsmuhio/MTGCN_Resources.
Enhancing temporal variability of 5-minute satellite-derived solar irradiance data

Satellite-derived solar irradiance data are known to underestimate temporal variability compared to point measurements because of their pixel-averaging nature. In this study, we apply an algorithm imposing random noise to enhance the temporal variability of 5-minute satellite-derived solar irradiance data. We show that the resulting product, termed as True Dynamics, has clear-sky exceedance events and the frequency of large ramp events closer to observation. In addition, the increase of temporal resolution of irradiance data significantly reduces the underestimation error of power inverter clipping under high DC:AC capacity ratios conditions.