Reviewerly/exHarmony huggingface dataset:
Roadkill: Nesting Laser-Cut Objects for Fast Assembly

We present Roadkill, a software tool that converts 3D models to 2D cutting plans for laser cutting—such that the resulting layouts allow for fast assembly. Roadkill achieves this by putting all relevant information into the cutting plan: (1) Thumbnails indicate which area of the model a set of parts belongs to. (2) Parts with exposed finger joints are easy to access, thereby suggesting to start assembly here. (3) Openings in the sheet act as jigs, affording assembly within the sheet. (4) Users continue assembly by inserting what has already been assembled into parts that are immediately adjacent or are pointed to by arrows. Roadkill maximizes the number of joints rendered in immediate adjacency by breaking down models into "subassemblies." Within a subassembly, Roadkill holds the parts together using break-away tabs. (5) Users complete subassemblies according to their labels 1, 2, 3…, following 1 -> 1 links to insert subassemblies into other subassemblies, until all parts come together. In our user study, Roadkill allowed participants to assemble layouts 2.4 times faster than layouts generated by a traditional pair-wise labeling of plates.
A Universal Quaternion Hypergraph Network for Multimodal Video Question Answering

Fusion and interaction of multimodal features are essential for video question answering. Structural information composed of the relationships between different objects in videos is very complex, which restricts understanding and reasoning. In this paper, we propose a quaternion hypergraph network (QHGN) for multimodal video question answering, to simultaneously involve multimodal features and structural information. Since quaternion operations are suitable for multimodal interactions, four components of the quaternion vectors are applied to represent the multimodal features. Furthermore, we construct a hypergraph based on the visual objects detected in the video. Most importantly, the quaternion hypergraph convolution operator is theoretically derived to realize multimodal and relational reasoning. Question and candidate answers are embedded in quaternion space, and a Q&A reasoning module is creatively designed for selecting the answer accurately. Moreover, the unified framework can be extended to other video-text tasks with different quaternion decoders. Experimental evaluations on the TVQA dataset and DramaQA dataset show that our method achieves state-of-the-art performance.
How Identification With the Social Environment and With the Government Guide the Use of the Official COVID-19 Contact Tracing App: Three Quantitative Survey Studies

Official contact tracing apps have been implemented and recommended for use across nations to track and contain the spread of COVID-19. Such apps can be effective if people are willing to use them. Accordingly, many attempts are being made to motivate citizens to make use of the officially recommended apps.The aim of this research was to contribute to an understanding of the preconditions under which people are willing to use a COVID-19 contact tracing app (ie, their use intentions and use). To go beyond personal motives in favor of app use, it is important to take people's social relationships into account, under the hypothesis that the more people identify with the beneficiaries of app use (ie, people living close by in their social environment) and with the source recommending the app (ie, members of the government), the more likely they will be to accept the officially recommended contact tracing app.Before, right after, and 5 months after the official contact tracing app was launched in Germany, a total of 1044 people participated in three separate surveys. Structural equation modeling was used to test the hypotheses, examining the same model in all studies at these critical points in time.Across the three surveys, both identification with the beneficiaries (people living in their social environment) and with the source recommending the app (members of the government) predicted greater intention to use and use (installation) of the official contact tracing app. Trust in the source (members of the government) served as a mediator. Other types of identification (with people in Germany or people around the world) did not explain the observed results. The findings were highly consistent across the three surveys.Attempts to motivate people to use new health technology (or potentially new measures more generally) not only for their personal benefit but also for collective benefits should take the social context into account (ie, the social groups people belong to and identify with). The more important the beneficiaries and the sources of such measures are to people's sense of the self, the more willing they will likely be to adhere to and support such measures.
Efficient Quadtree Search for HEVC Coding Units for V-PCC

Dynamic point clouds (DPC) are new media storage formats that allow end-users to watch objects/scenes in a three-dimensional (3D) sense.It can be displayed from different angles throughout time.However, the raw size of a point cloud is huge because there can be millions of points (each containing color triplet and location triplet information) in a point cloud, and there can be multiple point clouds in a DPC.Video-based point cloud compression (V-PCC) is developed to project a 3D point cloud to 2D images: attribute, geometry, and occupancy images.After padding, the 2D images are compressed using the wellestablished high-efficiency video coding (HEVC).In this study, we first employ an occupancy image to propose a blocky occupancy flag (BOF), to denote the occupancy information on "a block basis".For coding attribute and geometry images, we use a BOF to develop a fast coding unit (CU) algorithm for early termination of the CU search recursion.We also utilize the geometry images to calculate the 2D and 3D information of each pixel, for 2D/3D spatial homogeneity of the pixels to design fast CU decision.In addition, we proposed a modified rate-distortion optimization for different color components considering the picture order count (POC) structure in HEVC/V-PCC.Finally, we propose an HEVC input pixel modification method based on a BOF to reduce the unnecessary information to be coded for attribute images.Compared with the state-of-the-art fast V-PCC encoding method, the proposed work outperforms by up to 2.31% in Bjøntegaard delta bit rates (BDBR) (with very slight loss by only up to 0.38%), and improves the time saving performances by up to 7.84% for two different testing datasets. INDEX TERMSVideo-based point cloud compression (V-PCC), dynamic point cloud (DPC), High Efficiency Video Coding (HEVC), fast coding unit (CU) decision algorithm, occupancy map.
Predicting Gaze from Egocentric Social Interaction Videos and IMU Data

Gaze prediction in egocentric videos is a fairly new research topic, which might have several applications for assistive technology (e.g., supporting blind people in their daily interactions), security (e.g., attention tracking in risky work environments), education (e.g., augmented / mixed reality training simulators, immersive games) and so forth. Egocentric gaze is typically estimated from video while few works attempt to use inertial measurement unit (IMU) data, a sensor modality often available in wearable devices (e.g., augmented reality headsets). Instead, in this paper, we examine whether joint learning of egocentric video and corresponding IMU data can improve the first-person gaze prediction compared to using these modalities separately. In this respect, we propose a multimodal network and evaluate it on several unconstrained social interaction scenarios captured by a first-person perspective. The proposed multimodal network achieves better results compared to unimodal methods as well as several (multimodal) baselines, showing that using egocentric video together with IMU data can boost the first-person gaze estimation performance.
Quantum error correction with higher Gottesman-Kitaev-Preskill codes: Minimal measurements and linear optics

We propose two schemes to obtain Gottesman-Kitaev-Preskill (GKP) error syndromes by means of linear optical operations, homodyne measurements and GKP ancillae. This includes showing that for a concatenation of GKP codes with a $[n,k,d]$ stabilizer code only $2n$ measurements are needed in order to obtain the complete syndrome information, significantly reducing the number of measurements in comparison to the canonical concatenated measurement scheme and at the same time generalizing linear-optics-based syndrome detections to higher GKP codes. Furthermore, we analyze the possibility of building the required ancilla states from single-mode states and linear optics. We find that for simple GKP codes this is possible, whereas for concatenations with qubit Calderbank-Shor-Steane (CSS) codes of distance $d\geq3$ it is not. We also consider the canonical concatenated syndrome measurements and propose methods for avoiding crosstalk between ancillae. In addition, we make use of the observation that the concatenation of a GKP code with a stabilizer code forms a lattice in order to see the analog information decoding of such codes from a different perspective allowing for semi-analytic calculations of the logical error rates.
Explanation as a Process: User-Centric Construction of Multi-level and Multi-modal Explanations

In the last years, XAI research has mainly been concerned with developing new technical approaches to explain deep learning models. Just recent research has started to acknowledge the need to tailor explanations to different contexts and requirements of stakeholders. Explanations must not only suit developers of models, but also domain experts as well as end users. Thus, in order to satisfy different stakeholders, explanation methods need to be combined. While multi-modal explanations have been used to make model predictions more transparent, less research has focused on treating explanation as a process, where users can ask for information according to the level of understanding gained at a certain point in time. Consequently, an opportunity to explore explanations on different levels of abstraction should be provided besides multi-modal explanations. We present a process-based approach that combines multi-level and multi-modal explanations. The user can ask for textual explanations or visualizations through conversational interaction in a drill-down manner. We use Inductive Logic Programming, an interpretable machine learning approach, to learn a comprehensible model. Further, we present an algorithm that creates an explanatory tree for each example for which a classifier decision is to be explained. The explanatory tree can be navigated by the user to get answers of different levels of detail. We provide a proof-of-concept implementation for concepts induced from a semantic net about living beings.
GAMnet

Recently, deep graph matching (GM) methods have gained increasing attention. These methods integrate graph nodes¡¯s embedding, node/edges¡¯s affinity learning and final correspondence solver together in an end-to-end manner. For deep graph matching problem, one main issue is how to generate consensus node's embeddings for both source and target graphs that best serve graph matching tasks. In addition, it is also challenging to incorporate the discrete one-to-one matching constraints into the differentiable correspondence solver in deep matching network. To address these issues, we propose a novel Graph Adversarial Matching Network (GAMnet) for graph matching problem. GAMnet integrates graph adversarial embedding and graph matching simultaneously in a unified end-to-end network which aims to adaptively learn distribution consistent and domain invariant embeddings for GM tasks. Also, GAMnet exploits sparse GM optimization as correspondence solver which is differentiable and can also incorporate discrete one-to-one matching constraints approximately in natural in the final matching prediction. Experimental results on three public benchmarks demonstrate the effectiveness and benefits of the proposed GAMnet.
Secure and Efficiently Searchable IoT Communication Data Management Model: Using Blockchain as a New Tool

With the rapid development of the Internet of Things (IoT), more IoT devices are connected in the same network and communicate frequently for sharing data and switching instructions. So that the traditional centralized security architecture of IoT will be limited in terms of data storage space, data reliability, scalability, and operating costs. In this article, we propose a novel communication data management model based on blockchain technology. Specifically, all encrypted IoT communication data files are uploaded to the public cloud server for obtaining enough storage space, but the key information extracted from these data files (called “communication logs”) will be recorded in an “IoT ledger” maintained by all IoT devices. In fact, the IoT ledger is a kind of blockchain structured distributed database, which can solve the problem of data reliability, scalability and would not involve high operating costs. Besides, in order to efficiently search communication logs and not reveal any sensitive information of communication data, we design a secure search scheme over such blockchain database, the asymmetric scalar-product-preserving encryption (ASPE) approach is exploited to guarantee the data security, and the two layers index improves the search efficiency. Security analysis and experiments on synthetic data set show that our scheme is secure and efficient.
3D Multi-Object Tracking using Random Finite Set-based Multiple Measurement Models Filtering (RFS-M<sup>3</sup>) for Autonomous Vehicles

Multiple object tracking (MOT) is a critical module for enabling autonomous vehicles to achieve safe planing and navigation in cluttered environments. In tracking-by-detection systems, there are inevitably many false positives and misses among learning-based input detections. The challenge for MOT is to combine these detections into tracks, and filter them based on their uncertainties, states, and temporal consistency to achieve accurate and persistent tracks. In this paper, we propose to solve the 3D MOT problem for autonomous driving applications using a random finite set-based (RFS) Multiple Measurement Models filter (RFS-M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ). In partiuclar, we propose multiple measurement models for a Poisson multi-Bernoulli mixture (PMBM) filter in support of different application scenarios. Our RFS-M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> filter can naturally model these uncertainties accurately and elegantly. We combine the learning-based detections with our RFS-M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> tracker through incorporating the detection confidence score into the PMBM prediction and update step. The superior experimental results of our RFS-M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> tracker on Waymo, Argoverse and nuSceness datasets illustrate that our RFS-M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> tracker outperforms state-of-the-art deep learning-based and traditional filter-based approaches. To the best of our knowledge, this represents a first successful attempt for employing an RFS-based approach in conjunction with 3D learning-based amodal detections for 3D MOT applications with comprehensive validation using challenging datasets made available by industry leaders.
Optimal Path Planning for Intelligent UAVs Using Graph Convolution Networks

Unmanned Aerial Vehicles (UAVs) are in use for surveillance services in the geographic areas, that are very hard and sometimes not reachable by humans. Nowadays, UAVs are being used as substitutions to manned operations in various applications. The intensive utilization of autonomous UAVs has given rise to many new challenges. One of the vital problems that arise while deploying UAVs in surveillance applications is the Coverage Path Planning(CPP) problem. Given a geographic area, the problem is to find an optimal path/tour for the UAV such that it covers the entire area of interest with minimal tour length. A graph can be constructed from the map of the area under surveillance, using computational geometric techniques. In this work, the Coverage Path Planning problem is posed as a Travelling Salesperson Problem(TSP) on these graphs. The graphs obtained are large in number of vertices and edges and the real-time applications require good computation speed. Hence a model is built using Graph Convolution Network (GCN). The model is effectively trained with different problem instances such as TSP20, TSP50, and TSP100. Results obtained from the Concorde Benchmark Dataset were used to analyze the optimality of the predicted tour length by the GCN. The model is also evaluated against the performance of evolutionary algorithms on several self-constructed graphs. Particle Swarm Optimization, Ant Colony Optimization, and Firefly Algorithm are used to find optimal tours and are compared with GCN. It is found that the proposed GCN framework outperforms these evolutionary algorithms in optimal tour length and also the computation time.
Data Hiding in Neural Networks for Multiple Receivers [Research Frontier]

Recently, neural networks have become a promising architecture for some intelligent tasks. In addition to conventional tasks such as classification, neural networks can be used for data hiding. This paper proposes a data hiding scheme to transmit different data to multiple receivers via the same neural network simultaneously. Additional data are embedded into the neural network during the process of training instead of modifying the network parameters after training. As a result, the reduction in detection accuracy of the neural network is trivial with additional data embedded. On the receiver side, additional data can be extracted using specific data decoding networks. Instead of training, the parameters of data decoding networks are generated using the embedding keys possessed by the receivers. Therefore, it is unnecessary to store and transmit the decoding networks secretly, and the practicability is satisfactory. A receiver can extract corresponding additional data with the correct embedding key. For other parts of additional data, the receiver cannot affirm their existence and therefore cannot extract them. Experimental results verify the effectiveness of the proposed data hiding scheme, including embedding capacity and robustness.
Evolving hierarchical memory-prediction machines in multi-task reinforcement learning

A fundamental aspect of intelligent agent behaviour is the ability to encode salient features of experience in memory and use these memories, in combination with current sensory information, to predict the best action for each situation such that long-term objectives are maximized. The world is highly dynamic, and behavioural agents must generalize across a variety of environments and objectives over time. This scenario can be modeled as a partially-observable multi-task reinforcement learning problem. We use genetic programming to evolve highly-generalized agents capable of operating in six unique environments from the control literature, including OpenAI's entire Classic Control suite. This requires the agent to support discrete and continuous actions simultaneously. No task-identification sensor inputs are provided, thus agents must identify tasks from the dynamics of state variables alone and define control policies for each task. We show that emergent hierarchical structure in the evolving programs leads to multi-task agents that succeed by performing a temporal decomposition and encoding of the problem environments in memory. The resulting agents are competitive with task-specific agents in all six environments. Furthermore, the hierarchical structure of programs allows for dynamic run-time complexity, which results in relatively efficient operation.
Spatio-temporal Convolutional Attention Network for Spotting Macro- and Micro-expression Intervals

Emotional detection based on facial expressions is an important procedure in high-risk tasks such as criminal investigation or lie detection. To reduce the impact of the inconsistency in the duration of macro- and micro-expression, we propose an effective Spatio-temporal Convolutional Attention Network (STCAN) for spotting macro- and micro-expression intervals in long video sequences. The spatial features of each image in the video sequence are extracted through the Convolution Neural Network. Then, considering the problem of the inconsistency in the duration of the macro- and micro-expression, the multi-head self-attention model is used to analyze the weight of the spatial feature of the image in the temporal space. Finally, the time interval of emotional changes is determined according to the weight of each frame of the video sequence, and the macro- and micro-expression intervals are obtained through the threshold segmentation model. Considering the problem of Leave-One-Subject-Out cross-validation the training time long, we verified the effectiveness of our model on the SAMM Long Video and CAS(ME)2 datasets through the Leave-Half-Subject-Out (LHSO) cross-validation method. The experiments show that the STCAN model can achieve competitive results on Facial Micro-Expression (FME) Challenge 2021.
Automatic generation of Grover quantum oracles for arbitrary data structures

Abstract The steadily growing research interest in quantum computing—together with the accompanying technological advances in the realization of quantum hardware—fuels the development of meaningful real-world applications, as well as implementations for well-known quantum algorithms. One of the most prominent examples till today is Grover’s algorithm, which can be used for efficient search in unstructured databases. Quantum oracles that are frequently masked as black boxes play an important role in Grover’s algorithm. Hence, the automatic generation of oracles is of paramount importance. Moreover, the automatic generation of the corresponding circuits for a Grover quantum oracle is deeply linked to the synthesis of reversible quantum logic, which—despite numerous advances in the field—still remains a challenge till today in terms of synthesizing efficient and scalable circuits for complex Boolean functions. In this paper, we present a flexible method for automatically encoding unstructured databases into oracles, which can then be efficiently searched with Grover’s algorithm. Furthermore, we develop a tailor-made method for quantum logic synthesis, which vastly improves circuit complexity over other current approaches. Finally, we present another logic synthesis method that considers the requirements of scaling onto real world backends. We compare our method with other approaches through evaluating the oracle generation for random databases and analyzing the resulting circuit complexities using various metrics.