Reviewerly/exHarmony huggingface dataset:
Engagement Rewarded Actor-Critic with Conservative Q-Learning for Speech-Driven Laughter Backchannel Generation

We propose a speech-driven laughter backchannel generation model to reward engagement during human-agent interaction. We formulate the problem as a Markov decision process where speech signal represents the state and the objective is to maximize human engagement. Since online training is often impractical in the case of human-agent interaction, we utilize the existing human-to-human dyadic interaction datasets to train our agent for the backchannel generation task. We address the problem using an actor-critic method based on conservative Q-learning (CQL), that mitigates the distributional shift problem by suppressing Q-value over-estimation during training. The proposed CQL based approach is evaluated objectively on the IEMOCAP dataset for laughter generation task. When compared to the existing off-policy Q-learning methods, we observe an improved compliance with the dataset in terms of laugh generation rate. Furthermore, we show the effectiveness of the learned policy by estimating the expected engagement using off-policy policy evaluation techniques.
PyArmadillo: a streamlined linear algebra library for Python

PyArmadillo is a linear algebra library for the Python language, with the aim of closely mirroring the programming interface of the widely used Armadillo C++ library, which in turn is deliberately similar to Matlab. PyArmadillo hence facilitates algorithm prototyping with Matlab-like syntax directly in Python, and relatively straightforward conversion of PyArmadillo-based Python code into performant Armadillo-based C++ code. The converted code can be used for purposes such as speeding up Python-based programs in conjunction with pybind11, or the integration of algorithms originally prototyped in Python into larger C++ codebases. PyArmadillo provides objects for matrices and cubes, as well as over 200 associated functions for manipulating data stored in the objects. Integer, floating point and complex numbers are supported. Various matrix factorisations are provided through integration with LAPACK, or one of its high performance drop-in replacements such as Intel MKL or OpenBLAS. PyArmadillo is open-source software, distributed under the Apache 2.0 license; it can be obtained at https://pyarma.sourceforge.io or via the Python Package Index in precompiled form.
Digital Forensics, Video Forgery Recognition, for Cybersecurity Systems

The major advances in mobile video recording systems have led to an increased number of videos being streamed and uploaded on the internet daily. This has had a major impact on the expansion rate of the IoT with the addition of affordable security systems that capture and stream video content. These systems regularly meet minimal security standards resulting to breaches and data theft. The scientific branch of Computer Forensics has proposed various methods for protecting the confidentiality and integrity of videos, but the need for more robust and effective methods rises as videos grow, in size and number, while compression algorithms are becoming increasingly more efficient. The goal of this paper is to propose a new method on forgery detection, based on the characteristics of Dense Optical Flow. This method is applied on a low-cost device able to capture static CCTV video. The framework is tested for proof of its effectiveness in detecting copy-move, insertion and deletion forgeries. Ultimately, an identification automation method if proposed along with future improvements.
TAR-AFT: A Framework to Secure Shared Cloud Data with Group Management

In addition to replacing desktop-based methods, cloud computing is playing a significant role in several areas of data management. The health care industry, where so much data is needed to be handled correctly, is another arena in which artificial intelligence has a big role to play. The upshot of this innovation led to the creation of multiple healthcare clouds. The challenge of data privacy and confidentiality is the same for different clouds. Many existing works has provided security framework to ensure the security of data in clouds but still the drawback on revocation, resisting collusion attack along with privacy of data present a complex problem. For preserving the data privacy and confidentiality, a novel framework is proposed with two novel algorithms of Threat Aware Revocation (TAR) and Advance Flexi Twister Secret Block Encryption Standard (AFT-SBES) both are named as TAR-AFT. The TAR algorithm is mainly focus on generates the user signature and the AFT-SBES algorithm to generate the key. Both the signature and key are distributed to the users for enhancing the security to access cloud storage and files. The self-session shadow approach is used to monitor the activities of the cloud users. The revocation is carried out through removal of signature from the Enhanced Dynamic Hash Table (EDHT). From the performance analysis of TAR-AFT, it provides more effective to accessing of data stored in cloud and security such as data privacy and confidentiality.
Person Identification Based on Accelerations Sensed in Smartphones with LSTM

User identification is an important task for a variety of purposes such as authentication or providing personalized advice for improving user experience. In this paper, we propose a method for identifying a user who is holding a smartphone out of previously given target users from acceleration data obtained from the accelerometer in the smartphone using a deep neural network. This proposed method preliminary creates the model from the acceleration data of each user while walking in its training phase. This method identifies the user from acceleration data for identification based on this model in the classification phase. We evaluated the proposed method with the acceleration obtained from the actual eight and twelve users in two aspects, which were identifications including no-decision choice and that without no-decision choice. Our evaluation showed that the proposed method achieved accuracies higher than 95% for two- to five-class identification without no-decision. The proposed method identified the user with no or little false positive in evaluations with “no-decision.”
Can Optical Trojans Assist Adversarial Perturbations?

Recent work has demonstrated how physically realizable attacks on neural network vision pipelines can consistently produce misclassifications of a given target object. A smaller body of work has also produced modifications that can be applied directly to the neural network to generate incorrect predictions. However, although these perturbations are difficult to detect from examining the resulting images themselves, they are obvious if any testing is done on the network to check its accuracy. Here, we combine methods from both these lines of work to generate attacks that can be switched on or off. Specifically, we simulate a physically realizable Trojaned lens to attach to a camera that only causes the neural network vision pipeline to produce incorrect classifications if a specific adversarial patch is present in the scene. This novel Optical Trojan is used to amplify the effect of the adversarial patch so that we can achieve similar attack performance with smaller and less noticeable patches. To improve the robustness of our proposed method, we take into account the fabrication process with quantized lens parameters, deal with lens defocus using kernel scaling, and make it resilient against noise caused by the camera sensor readouts and test in various simulated settings. Finally, we propose a simple yet effective approach to detect such Trojaned lenses by analyzing the distributions of benign and Trojaned kernels.
Semi‐automatic reconstruction of object lines using a smartphone’s dual camera

Abstract In this paper, the possibility of reconstructing object lines using a smartphone’s rear dual camera (wide‐angle and telephoto) was examined through designing a semi‐automatic system. After calibrating both cameras, six scenes for each of three objects were captured and rectified. Object lines were categorised into six groups based on the distance and angle to the dual camera system. Image lines were extracted using the linear Hough transform technique and points of intersection detected. Stereo pairing of conjugate points then allowed the calculation of object coordinates and the lengths of object lines were compared to their lengths measured by a digital caliper. The best line reconstruction results were achieved with the smallest distance and angle to the dual camera system.
Vision Based Real-time High-accuracy Automatic Counting with Applications for Smart Pharmacy

At present, many industrial applications urgently need a real-time high-accuracy automatic counting algorithm to realize the counting function of various objects. Some scholars are studying the use of deep learning algorithms to count objects in industrial applications. However, facing closely arranged and different objects, deep learning algorithms cannot achieve this function. Aiming at the application background of regular cuboids counting with a wide range of requirements, this paper provides a real-time high-accuracy automatic counting algorithm based on a deep understanding of the scene and a comprehensive use of multiple computer vision technologies. The algorithm can meet the stringent requirements of industrial applications and realize real-time high-accuracy automatic counting of closely arranged cuboids. And there is no requirement for whether the surfaces of the cuboids are the same, and there is no need to prepare the template pictures of the cuboids in advance. The algorithm mainly uses the characteristics of gaps between cuboids, and adopts improved Hough line detection to identify gaps, and finally realizes the counting function. The algorithm proposed in this paper has been verified on the counting of pillboxes in the industrial application of smart pharmacies. Compared with the deep learning algorithm, it has the characteristics of good interpretability, good generalization, fast speed and high accuracy.
Zero-Shot Learning Based on Quality-Verifying Adversarial Network

Recently, generative adversarial network (GAN)-based zero-shot learning methods have attracted widespread attention. However, due to the randomness of GAN generation, most existing methods cannot well guarantee to generate sufficiently reliable features and have good generalization ability. Targeting at these problems, we propose an effective Quality-Verifying Adversarial Network (QVAN) that consists of one generator and double discriminators. Adversarial learning between the former discriminator and generator is to generate visual features, which can be partitioned into pseudo-generated features and reliable-generated features. The latter discriminator is used for quality-verifying that will guide the generator to generate more reliable features that are near the real visual features. To avoid over-fitting and ensure intra-class diversity, we set the threshold for each class to distinguish pseudo-generated features and reliable-generated features. To further preserve both compactness and discriminability of the samples, we introduce the class metric constraint, which are more conducive to classification. Moreover, we introduce <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{1,2}$</tex-math></inline-formula> -norm constraint to fully consider the specific distribution among different classes, thus making the generated features more discriminant. Extensive experiments on several real-world datasets show the effectiveness of the proposed approach, which demonstrate the advantage over the state-of-the-art methods.
A Deep Transfer Learning Method for Medical Question Matching

Question matching (QM) is a fundamental task of information retrieval (IR)-based question-answering (QA) systems, which can be formulated as a paraphrase identification (PI) problem and relies on large-scale labeled data, which is not easy to be obtained, especially in specific domains such as the medical domain. In this paper, we investigate transfer learning for QM in the medical domain, aiming to adapt the shared knowledge learnt from questions for one disease to other diseases. We first compare six state-of-the-art deep learning methods for QM in Chinese on different diseases, and then apply a transfer learning framework to these deep learning methods. Experiments on a Chinese medical corpus of three diseases about children show that the proposed transfer learning framework is efficient and can bring stable performance improvement for most deep learning based QM models on small-scale medical corpora of different diseases.
The Predictive Power of Spatial Relational Reasoning Models: A New Evaluation Approach

In the last few decades, cognitive theories for explaining human spatial relational reasoning have increased. Few of these theories have been implemented as computational models, however, even fewer have been compared computationally to each other. A computational model comparison requires, among other things, a still missing quantitative benchmark of core spatial relational reasoning problems. By presenting a new evaluation approach, this paper addresses: (1) developing a benchmark including raw data of participants, (2) reimplementation, adaptation, and extension of existing cognitive models to predict individual responses, and (3) a thorough evaluation of the cognitive models on the benchmark data. The paper shifts the research focus of cognitive modeling from reproducing aggregated response patterns toward assessing the predictive power of models for the individual reasoner. It demonstrate that not all psychological effects can discern theories. We discuss implications for modeling spatial relational reasoning.
Autonomous Vehicles, Moral Hazards &amp;amp; the "AV Problem"

The autonomous vehicle (“AV”) industry faces the following ethical question: “How do we know when our AV technology is safe enough to deploy at scale?” The search for an answer to this question is the “AV Problem.” This essay examines that question through the lens of the July 15, 2021 filing on Form S-4 with the Securities and Exchange Commission in the going public transaction for Aurora Inventions, Inc.The filing reveals that successful implementation of Aurora’s business plan in the long term depends on the truth of the following proposition: A vehicle controlled by a machine driver is safer than a vehicle controlled by a human driver (the “Safety Proposition”).In a material omission for which securities law liability may attach, the S-4 fails to state Aurora’s position on deployment: will Aurora delay deployment until such time as it believes the Safety Proposition is true to a reasonable certainty or will it deploy at scale earlier in the hope that increased current losses will be offset by anticipated future safety gains?The Safety Proposition is a statement about physical probability which is either true or false. For success, AV companies need the public to believe the Safety Proposition, yet belief is not the same as truth. The difference between truth and belief creates tension in the S-4 because the filing both fosters a belief in the Safety Proposition while at the same time making clear there is insufficient evidence to support the truth of the Safety Proposition.A moral hazard results when financial pressures push for early deployment of AV systems before evidence shows that the Safety Proposition is true to a reasonable certainty. This problem is analyzed by comparison with the famous trolley problem in ethics and consideration of corporate governance techniques which an AV company might use to ensure the integrity of its decision process for deployment. The AV industry works to promote belief in the safety proposition in the hope that the public will accept that AV technology has benefits, thus avoiding the need to confront the truth of the Safety Proposition directly. This hinders a meaningful public debate about the merits and timing of deployment of AV technology, raising the question of whether there is a place for meaningful government regulation.
Reproducibility Companion Paper

In our MM'20 paper,, we presented a Kalman filter-based approach for prediction of head motion in 6DoF. The proposed approach was employed in our cloud-based volumetric video streaming system to reduce the interaction latency experienced by the user. In this companion paper, we present the dataset collected for our experiments and our simulation framework that reproduces the obtained experimental results. Our implementation is freely available on Github to facilitate further research.
Hybrid and Ensemble-Based Personalized Recommender System - Solving Data Sparsity Problem

Online content streaming is the most popular form of entertainment in recent times due to COVID 19 lockdown. All popular streaming services use various product recommendation schemes to retain users to their services by intriguing them with content that they might like. Various recommendation systems have been used by famous streaming services like Netflix, Amazon Prime, Hulu, etc. but they lack consistency and accuracy as they suffer from some severe problems such as the first rater problem, sparsity problem, and various computations problems. In this research, we have come up with a hybrid machine learning recommender system which uses an ensemble of content-based and collaborative filtering techniques to not only solve all data sparsity problems but also provide more personalized recommendations to the users based on their watching history and user profile. This research provides a new algorithm that increases the quality of content that is being recommended to the users.
Airplane Detection and Recognition Incorporating Target Component Detection

In the 2020 Gaofen Challenge on Automated High- Resolution Earth Observation Image Interpretation [1], there is a topic on airplane detection and recognition in optical images. The task is to detect and classify 10 types of civil airplanes, and the difficulty lies in the classification of similar airplanes. We propose a novel airplane detection and recognition method incorporating component detection, which is based on data preprocessing, basic detection network and object head detection. With the experiments between our approach and other detection networks in local training dataset, online validation and test dataset, our method effectively improved the performance of detection and recognition of airplanes in optical images.
A Virtual Character Generation and Animation System for E-Commerce Live Streaming

Virtual character has been widely adopted in many areas, such as virtual assistant, virtual customer service, robotics and etc. In this paper, we focus on its application in e-commerce live streaming. Particularly, we propose a virtual character generation and animation system that supports e-commerce live streaming with virtual characters as anchors. The system offers a virtual character face generation tool based on a weakly supervised 3D face reconstruction method. The method takes a single photo as input and generates a 3D face model with both similarity and aesthetics considered. It does not require 3D face annotation data due to the assist of differentiable neural rendering technique which seamlessly integrates rendering into a deep learning based 3D face reconstruction framework. Moreover, the system provides two animation approaches which support two different ways of live stream respectively. The first approach is based on real-time motion capture. An actor's performance is captured in real-time via a monocular camera, and then utilized for animating a virtual anchor. The second approach is text driven animation, in which the human-like animation is automatically generated based on a text script. The relationship between text script and animation is learned based on the training data which can be accumulated via the motion capture based animation. To our best knowledge, the presented work is the first sophisticated virtual character generation and animation system that is designed for e-commerce live streaming and actually deployed on an online shopping platform with millions of daily audiences.
Three-Axis Wireless Gesture-Controlled Car

A novel approach of three-axis gesture-controlled car has been illustrated in this paper. The propose scheme consists of three design modules: transmitter, receiver and the wireless network. The hand glove is integrated with a three-axis digital accelerometer ADXL 345 and NodeMCU-based transmitter unit which sends the reference command to the car unit associated with a Raspberry Pi-based receiver module. A Raspberry Pi 3B + module is used for the same. Human data glove is developed by incorporating six different human hand movements by using the accelerometer. Different features are obtained from the hand gesture data using statistical formulas. The car is driven using Raspberry Pi. Wireless application is based upon MQTT protocol. The wireless network is developed with same gateway address in which all the devices are connected. Hardware prototype of the proposed method is shown here.
Adaptive Reset Techniques for Haptic Retargeted Interaction

This article presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors.
ARGUABLY @ AI Debater-NLPCC 2021 Task 3: Argument Pair Extraction from Peer Review and Rebuttals

This paper describes our participating system run to the argumentative text understanding shared task for AI Debater at NLPCC 2021 (http://www.fudan-disc.com/sharedtask/AIDebater21/tracks.html). The tasks are motivated towards developing an autonomous debating system. We make an initial attempt with Track-3, namely, argument pair extraction from peer review and rebuttal where we extract arguments from peer reviews and their corresponding rebuttals from author responses. Compared to the multi-task baseline by the organizers, we introduce two significant changes: (i) we use ERNIE 2.0 token embedding, which can better capture lexical, syntactic, and semantic aspects of information in the training data, (ii) we perform double attention learning to capture long-term dependencies. Our proposed model achieves the state-of-the-art results with a relative improvement of 8.81% in terms of F1 score over the baseline model. We make our code available publicly at https://github.com/guneetsk99/ArgumentMining_SharedTask. Our team ARGUABLY is one of the third prize-winning teams in Track 3 of the shared task.