Reviewerly/exHarmony huggingface dataset:
Design of multiple artificial potential function and selector function for autonomous driving vehicle

This study proposed a novel method for designing artificial potential functions (APFs) for autonomous driving vehicle. In this method, a control performance index is firstly designed that reflects the control requirements. In practical use, a suitable APF should be selected from the prepared candidate APFs. In designing an APF selector function, it is necessary to include the value of performance index in the learning process to exclude dangerous choice of candidate APF caused by misclassification. It can be realized by defining the novel error function in the learning process of neural networks. The usefulness of the proposed method is verified through numerical experiments.
Story segmentation for news broadcast based on primary caption

In the information explosion era, people only want to access the news information that they are interested in. News broadcast story segmentation is strongly needed, which is an essential basis for personalized delivery and short video. The existing advanced story boundary segmentation methods utilize semantic similarity of subtitles, thus entailing complex semantic computation. The title texts of news broadcast programs include headline (or primary) captions, dialogue captions and the channel logo, while the same story clips only render one primary caption in most news broadcast. Inspired by this fact, we propose a simple method for story segmentation based on the primary caption, which combines YOLOv3 based primary caption extraction and preliminary location of boundaries. In particular, we introduce mean hash to achieve the fast and reliable comparison for detected small-size primary caption blocks. We further incorporate scene recognition to exact the preliminary boundaries, because the primary captions always appear later than the story boundary. Experimental results on two Chinese news broadcast datasets show that our method enjoys high accuracy in terms of R, P and F1-measures.
Manipulation of the asymmetric Einstein–Podolsky–Rosen steering via coherent population transfer

Abstract We investigate the Einstein–Podolsky–Rosen (EPR) steering of two cavity modes generated via four-wave mixing processes. For a three-level V-type configuration, a strong coherent field as a dressing field couples the atoms, and a weak coherent field as a control filed resonantly couples one dressed state to the auxiliary state. By using the approach of dressed states and Bogoliubov modes, we find that only one Bogoliubov mode is mediated into the interaction with the dressed atoms, and the other one is decoupled. Due to the presence of the coherent control filed, the channel for dressed state population transfer is established, which plays a key role in the generation of the asymmetric EPR steering of two cavity fields. It is shown that the one-way EPR steering can be manipulated by the Rabi frequencies of the control field and the cooperativity parameters.
Trainable Spectrally Initializable Matrix Transformations in Convolutional Neural Networks

In this work, we introduce a new architectural component to Neural Network (NN), i.e., trainable and spectrally initializable matrix transformations on feature maps. While previous literature has already demonstrated the possibility of adding static spectral transformations as feature processors, our focus is on more general trainable transforms. We study the transforms in various architectural configurations on four datasets of different nature: from medical (ColorectalHist, HAM10000) and natural (Flowers) images to historical documents (CB55). With rigorous experiments that control for the number of parameters and randomness, we show that networks utilizing the introduced matrix transformations outperform vanilla neural networks. The observed accuracy increases appreciably across all datasets. In addition, we show that the benefit of spectral initialization leads to significantly faster convergence, as opposed to randomly initialized matrix transformations. The transformations are implemented as auto-differentiable PyTorch modules that can be incorporated into any neural network architecture. The entire code base is open-source.
Video reconstruction by spatio-temporal fusion of blurred-coded image pair

Learning-based methods have enabled the recovery of a video sequence from a single motion-blurred image or a single coded exposure image. Recovering video from a single motion-blurred image is a very ill-posed problem and the recovered video usually has many artifacts. In addition to this, the direction of motion is lost and it results in motion ambiguity. However, it has the advantage of fully preserving the information in the static parts of the scene. The traditional coded exposure framework is better-posed but it only samples a fraction of the space-time volume, which is at best 50% of the space-time volume. Here, we propose to use the complementary information present in the fully-exposed (blurred) image along with the coded exposure image to recover a high fidelity video without any motion ambiguity. Our framework consists of a shared encoder followed by an attention module to selectively combine the spatial information from the fully-exposed image with the temporal information from the coded image, which is then super-resolved to recover a non-ambiguous high-quality video. The input to our algorithm is a fully-exposed and coded image pair. Such an acquisition system already exists in the form of a Coded-two-bucket (C2B) camera. We demonstrate that our proposed deep learning approach using blurred-coded image pair produces much better results than those from just a blurred image or just a coded image.
Learning to Drive with Deep Reinforcement Learning

Autonomous driving cars are important due to improved safety and fuel efficiency. Various techniques have been described to consider only a single task, for example, recognition, prediction, and planning with supervised learning techniques. Some limitations of previous studies are: (1) human bias from human demonstration; (2) the need for multiple components such as localization, road mapping etc. with a complicated fusion logic; (3) in reinforcement learning, the focus was mostly on the learning algorithms but less on the evaluation of different sensors and reward functions. We describe end-to-end reinforcement learning for an autonomous car, which used only a single reinforcement learning model to create the autonomous car. Further, we designed a new efficient reward function to make the agent learn faster (18% improvement for all settings compared to the baseline reward function) and build the car with only the necessary perceptions and sensors. We show that it performed better with state-of-the-art off-policy reinforcement learning for continuous action (SAC, TD3).
Replication Package for Representation of Developer Expertise in Open Source Software

This describes the artifact associated with the article "Representation of Developer Expertise in Open Source Software" at the International Conference on Software Engineering 2021. The aim of the original paper was to define afeasible representation of adeveloper's expertise in specific focus areas of software development by gauging their fluency with different sets of APIs. The artifact is made available through Zenodo under the CC-BY-4.0 license at https://doi.org/10.5281/zenodo.4457107. The README file has detailed instructions on how to replicate the results presented in the original paper. The artifact includes the input dataset (with the developers' names and email addresses replaced by their corresponding SHA1 digest values to protect privacy) and all the associated scripts. The trained Doc2Vec models are also included in the artifact. These models can be used to obtain the Skill Space representations of developers, projects, and APIs without having to re-train the model.
SIGCHI Outstanding Dissertation Award: Profiling Artificial Intelligence as a Material for User Experience Design

HCI has become especially interested in the promises and challenges of user experience of AI, such as user acceptance, human-agent teamwork, and accessibility. Less discussed, however, is that the field of HCI routinely grapples with such challenges; across the various technologies commonly referred to as AI (e.g., predictive modeling, computer vision, NLP), what shared characteristics made human-AI interaction appeared uniquely difficult to design in the first place? Synthesizing my hands-on design and research over the past six years, in my dissertation, I worked to articulate whether, why, and how human-AI interaction appears uniquely challenging to design with established HCI methods. In this extended abstract, I first describe a human-AI interaction design framework as an answer to this question. I then discuss one critical implication of this framework: Framing data-driven AI systems as living socio-technical systems that co-evolve with their users. I analogize this reframing to the shift from desktop computing to ubiquitous computing and outline the ethnographic, design, and technological research opportunities it reveals.
Designing Human–Computer Integration in an Exertion Context

Human–computer interaction researchers continue to support the exerting user to promote the many benefits of being physically active. Recently, due to technological advances, systems emerged that can continuously sense, interpret and automatically act on information, opening the opportunity to design human-computer "integration", where the user and the system work in a partnership. However, today's knowledge gap is how to design human–computer integration experiences in an exertion context. In this talk, I present how I aimed to close this gap through the design of three eBike systems that "act on" different data types (motion, traffic light data, and EEG (electroencephalography)) to explore integrations with the exerting body. The investigation informed the creation of the "integrated exertion framework", which can guide interaction designers on how to, and in an inclusive manner, amplify a person's sensations of their abilities in an exertion context to create "superpower", "co-operative" and "symbiotic" human-computer partnerships.
Scene Classification Algorithm Based on Semantic Segmented Objects

Scene classification is one of the important problems in image and video search and recommendation systems. We propose an efficient scene classification method for three different classes by detecting objects in the scene. For detecting objects in an image, pre-trained semantic segmentation model is used. A weight matrix which has bias values to determine a scene class statistically is constructed. Finally, we classify an image into one of three classes (i.e. indoor, nature, city) by using the designed weighting matrix. The proposed method achieved 92% of verification accuracy and improved over 2% when comparing to the existing convolutional neural network (CNN) models.
Anomaly Detection on Time-series Logs for Industrial Network

With the deep integration of industrialization and informatization, the network environment is becoming more and more complex, and security is facing a huge threat. Recently, the industrial control systems pose an open trend, so the strategy of preventing external attacks through “physical isolation” does not work anymore. The security threats in the traditional IT field gradually affect the security of industrial control networks. Recently, more and more researchers apply artificial intelligence algorithms and blockchain technology to industrial control network security. This paper aims to propose a new way of thinking, starting from two levels of physical topology and time series structure for a specific industrial control system, establish a graph data structure, and then use the graph neural network (GNN) algorithm to detect abnormal nodes. We evaluate our approach through comprehensive experiments and the results are promising.
A large-scale image retrieval system for everyday scenes

We present a system for large-scale image retrieval on everyday scenes with common objects. Our system leverages advances in deep learning and natural language processing (NLP) for improved understanding of images by capturing the relationships between the objects within an image. As a result, a user can retrieve highly relevant images and obtain suggestions for similar image queries to further explore the repository. Each image in the repository is processed (using deep learning) to obtain the most probable captions and objects in it. The captions are parsed into tree structures using NLP techniques, and stored and indexed in a database system. When a query image is posed, an optimized tree-pattern query is executed by the database system to obtain candidate matches, which are then ranked using tree-edit distance of the tree structures to output the top-k matches. Word embeddings and Bloom filters are used to obtain similar image queries. By clicking the suggested similar image queries, a user can intuitively explore the repository.
Visiting-Route Recommendation in Amusement Parks and its Evaluations by an Ising Machine

In an amusement park, an attraction-visiting route considering the waiting time and traveling time improves visitors' satisfaction and experience. We focus on Ising machines to solve the problem, which are recently expected to solve combinatorial optimization problems at high speed by mapping the problems to Ising models or quadratic unconstrained binary optimization (QUBO) models. We propose a mapping of the visiting-route recommendation problem in amusement parks to a QUBO model for solving it using Ising machines. By using an actual Ising machine, we could obtain feasible solutions 15 times faster with almost the same accuracy as the simulated annealing method for the visiting-route recommendation problem.
Adversarial Black-Box Attacks on Vision-based Deep Reinforcement Learning Agents

Deep reinforcement learning (DRL) has had a profound impact in the field of robotic learning, especially in vision-based end-to-end applications. To ensure the robustness and stability of such robotic systems, their vulnerability to possible adversarial attacks must be explored. In this paper we demonstrate the first realistic black-box attack on vision-based DRL systems by adopting the concept of adversarial patches. Agents trained to perform object grasping based only on visual input are manipulated through their observation space. Different patch sizes and positions for attacks targeting DRL systems in simulation are evaluated, which prove to be decisive factors that greatly influence the effectiveness of adversarial patch attacks. Despite the fact that evaluated adversarial patches take up less than 2% of the images observed, the attacked DRL systems are heavily affected and show performance drop of up to 99 %. Our experiments and results in the simulation pave the way for more realistic adversarial attacks on DRL agents.
Densely Connected Graph Attention Network Based on Iterative Path Reasoning for Document-Level Relation Extraction

Document-level relation extraction is a challenging task in Natural Language Processing, which extracts relations expressed with one or multiple sentences. It plays an important role in data mining and information retrieval. The key challenge comes from the indirect relations expressed across sentences. Graph-based neural networks have been proved effective for modeling structural information among the document. Existing methods enhance the graph models by using either the attention mechanism or the iterative path reasoning, which is not enough to capture all the effective structural information. In this paper, we propose a densely connected graph attention network based on iterative path reasoning (IPR-DCGAT) for document-level relation extraction. Our approach uses densely connected graph attention network to model the local and global information among the document. In addition, we propose to learn dynamic path weights for reasoning relations across sentences. Extensive experiments on three datasets demonstrate the effectiveness of our approach. Our model achieves 84% F1 score on CDR, which is about 16.3%–22.5% higher than previous models with a significant margin. Meanwhile, the results of our approach are also comparably superior to the state-of-the-art results on the GDA and DocRED dataset.
Filter-Wise Quantization of Deep Neural Networks for IoT Devices

Network quantization is an effective compression technique of deep neural networks (DNNs) for on-device machine learning at consumer devices. Existing layer-wise quantization techniques allocate different bitwidths to different network layers. In this paper, we propose a filter-wise quantization technique based on the differentiable neural architecture search (DNAS). We use a two-level network structure and a novel candidate generation algorithm, which can substantially prune the large search space. The effectiveness of our technique was validated with MobileNetV2 on ImageNet.
Detecting Manipulated Facial Videos: A Time Series Solution

We propose a new method to expose fake videos based on a time series solution. The method is based on bidirectional long short-term memory (Bi-LSTM) backbone architecture with two different types of features: Face-Alignment and Dense-Face-Alignment, in which both of them are physiological signals that can be distinguished between fake and original videos. We choose 68 landmark points as the feature of Face-Alignment and Pose Adaptive Feature (PAF) for Dense-Face-Alignment. Based on these two facial features, we designed two deep networks. In addition, we optimize our network by adding an attention mechanism that improves detection precision. Our method is tested over benchmarks of Face Forensics/Face Forensics++ dataset and show a promising performance on inference speed while maintaining accuracy with state-of art solutions that deal against DeepFake.
Real Time Face Recognition based on Convolution Neural Network

: Face recognition is an active research area with many successful traditional and deep learning methods. In our research, we are introducing a Real Time Face Recognition based on Convolution Neural Network (CNN) method for face recognition and to maintain a high accuracy which is running in real time.Face represents multidimensional, complex, meaningful visualization and developing a real time face recognition computational model for face recognition is not easy. In present time, real time face recognition is a challenging problem. The main challenge of face recognition is that how to improve the image recognition performance of the system when it is affected by the variability of non-linear effects. In this paper, 3-layer CNN architecture is proposed for solving the face recognition problem, CNN gives a solution which is capable of solving the facial images related problems that contains varying illumination, poses, occlusions and facial expressions.The test on 115 subject of Essex University database achieve an accuracy of 98.5% which is similar range of performance as the best result of previous works. Meaningfully, our system of real time face recognition based on CNN completes the facial recognition process in less than 0.019 seconds.
Multi-resolution Wavelet Neural Network Learning Algorithm Based on Artificial Fish Swarm Algorithm

For the shortcoming on the algorithm of Wavelet Neural Network (WNN), a WNN learning algorithm based on Artificial Fish Swarm Algorithm (AFSA) is proposed. In the method, AFSA is used to train the WNN model. The simulation result show that, the proposed method has a good ability with features of modeling accuracy, training time, controlling precocity and local convergence, and it is also effective and feasible for the modeling of nonlinear complex system.
Design and Analysis of Intelligent Teaching System in Colleges and Universities Based on Big Data Application

Application based on big data, Around the teaching process, a set of intelligent, paperless information and automation efficient and intelligent teaching system is formed for college students from before class to after class. The system analyzed and designed in detail the function of face checking attendance based on artificial intelligence and GPS positioning, the function of fun teaching interaction, the function of intelligent online examination, and the function of automatic intelligent data statistics. It aims to improve teaching efficiency and students' learning initiative, and meet the needs of The Times, society and industry under the new situation of development.
Syntax Splitting for Iterated Contractions, Ignorations, and Revisions on Ranking Functions Using Selection Strategies

For characterizing belief sets consisting of independent parts, Parikh introduced the notion of syntax splitting. Corresponding postulates have been developed for the reasoning from and for the revision of belief bases with respect to syntax splitting. Kern-Isberner and Brewka introduced syntax splitting for epistemic states and iterated belief revision. Only recently, syntax splitting has also been studied for contractions and iterated contractions of epistemic states; however, all of the evaluated contractions proposed in the literature failed to fulfil the full syntax splitting postulates. In this paper, we study syntax splitting for iteratively contracting and revising epistemic states, represented by ranking functions, not only with respect to a set of formulas, but with respect to a set of conditionals. Using a framework of belief change governed by the principle of conditional preservation, we employ the concept of selection strategies. We develop axioms for selection strategies ensuring that the induced contractions and revisions fully obey the desired syntax splitting properties. Furthermore, we transfer our approach to ignorations and prove a theorem showing how selection strategies satisfying the axioms can effectively be constructed.
Self-supervised Graph Representation Learning with Variational Inference

Graph representation learning aims to convert the graph-structured data into a low dimensional space in which the graph structural information and graph properties are maximumly preserved. Graph Neural Networks (GNN)-based methods have shown to be effective in dealing with the graph representation learning task. However, most GNN-based methods belong to supervised learning, which depends heavily on the data labels that are difficult to access in real-world scenarios. In addition, the inherent incompleteness in data will further degrade the performance of GNN-based models. In this paper, we propose a novel self-supervised graph representation learning model with variational inference. First, we strengthen the semantic relation between node and graph level in a self-supervised manner to alleviate the issue of over-dependence on data labels. Second, we utilize the variational inference technique to capture the general pattern underlying the data, thus guaranteeing the model robustness under some data missing circumstances. Extensive experiments on three widely used citation network datasets show that our proposed method has achieved or matched state-of-the-art results on link prediction and node classification tasks.