Reviewerly/exHarmony huggingface dataset:
Fractional Motion Estimation for Point Cloud Compression

Motivated by the success of fractional pixel motion in video coding, we explore the design of motion estimation with fractional-voxel resolution for compression of color attributes of dynamic 3D point clouds. Our proposed block-based fractional-voxel motion estimation scheme takes into account the fundamental differences between point clouds and videos, i.e., the irregularity of the distribution of voxels within a frame and across frames. We show that motion compensation can benefit from the higher resolution reference and more accurate displacements provided by fractional precision. Our proposed scheme significantly outperforms comparable methods that only use integer motion. The proposed scheme can be combined with and add sizeable gains to state-of-the-art systems that use transforms such as Region Adaptive Graph Fourier Transform and Region Adaptive Haar Transform.
Language-Independent Speaker Anonymization Approach Using Self-Supervised Pre-Trained Models

Speaker anonymization aims to protect the privacy of speakers while preserving spoken linguistic information from speech. Current mainstream neural network speaker anonymization systems are complicated, containing an F0 extractor, speaker encoder, automatic speech recognition acoustic model (ASR AM), speech synthesis acoustic model and speech waveform generation model. Moreover, as an ASR AM is language-dependent, trained on English data, it is hard to adapt it into another language. In this paper, we propose a simpler self-supervised learning (SSL)-based method for language-independent speaker anonymization without any explicit language-dependent model, which can be easily used for other languages. Extensive experiments were conducted on the VoicePrivacy Challenge 2020 datasets in English and AISHELL-3 datasets in Mandarin to demonstrate the effectiveness of our proposed SSL-based language-independent speaker anonymization method.
Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation

End-to-end speech-to-speech translation (S2ST) without relying on intermediate text representations is a rapidly emerging frontier of research. Recent works have demonstrated that the performance of such direct S2ST systems is approaching that of conventional cascade S2ST when trained on comparable datasets. However, in practice, the performance of direct S2ST is bounded by the availability of paired S2ST training data. In this work, we explore multiple approaches for leveraging much more widely available unsupervised and weakly-supervised speech and text data to improve the performance of direct S2ST based on Translatotron 2. With our most effective approaches, the average translation quality of direct S2ST on 21 language pairs on the CVSS-C corpus is improved by +13.6 BLEU (or +113% relatively), as compared to the previous state-of-the-art trained without additional data. The improvements on low-resource language are even more significant (+398% relatively on average). Our comparative studies suggest future research directions for S2ST and speech representation learning.
Online-Updated High-Order Collaborative Networks for Single Image Deraining

Single image deraining is an important and challenging task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. Most of the existing deep-learning-based methods constrain the network to generate derained images but few of them explore features from intermediate layers, different levels, and different modules which are beneficial for rain streaks removal. In this paper, we propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module to exploit features from deep networks externally and internally for rain streaks removal. Externally, we design a deraining framework with three sub-networks trained in a collaborative manner, where the bottom network transmits intermediate features to the middle network which also receives shallower rainy features from the top network and sends back features to the bottom network. Internally, we enforce multi-scale compact constraints on the intermediate layers of deep networks to learn useful features via a Laplacian pyramid. Further, we develop a bidirectional scale-content similarity mining module to explore features at different scales in a down-to-up and up-to-down manner. To improve the model performance on real-world images, we propose an online-update learning approach, which uses real-world rainy images to fine-tune the network and update the deraining results in a self-supervised manner. Extensive experiments demonstrate that our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic datasets and one real-world dataset.
BVI-CC: A Dataset for Research on Video Compression and Quality Assessment

The video technology scenery has been very vivid over the past years, with novel video coding technologies introduced that promise improved compression performance over state-of-the-art technologies. Despite the fact that a lot of video datasets are available, representative content of the wide parameter space along with subjective evaluations of variations of encoded content from an unpartial end is required. In response to this requirement, this paper features a dataset, the BVI-CC. Three video codecs were deployed to create the variations of the encoded sequences: High Efficiency Video Coding (HEVC) Test Model (HM), AOMedia Video 1 (AV1), and Versatile Video Coding (VVC) Test Model (VTM). Nine source video sequences were carefully selected to offer both diversity and representativeness in the spatio-temporal domain. Different spatial resolution versions of the sequences were created and encoded by all three codecs at pre-defined target bit rates. The compression efficiency of the codecs was evaluated with commonly used objective quality metrics, and the subjective quality of their reconstructed content was also evaluated through psychophysical experiments. Additionally, an adaptive bit rate (convex hull rate-distortion optimization across spatial resolutions) test case was assessed using both objective and subjective evaluations. Finally, the computational complexities of the tested codecs were examined. All data have been made publicly available as part of the dataset, which can be used for coding performance evaluation and video quality metric development.
AI intelligent wayfinding based on Unreal Engine 4 static map

Abstract The development of games is closely related to the development of game engines. Unreal Engine is a mainstream game development engine. Unreal Engine has a powerful lighting rendering function and blueprint programming system.Therefore, using Unreal Engine for game development will better meet the requirements of modern game users for game quality and game content. In games, the AI path-finding system is often the core of a game. However, the path-finding component of the unreal engine is usually only applies to three-dimensional game and is used for intelligent path-finding in dynamic environment. Howerve, the 2D game usually finds the way in static environment. At present, there are a lareg number of path-finding algorithms and theoretical studies based on static environment, but there is no simulation experiment and detailed explanation of intelligent path-finding in static environment based on the Unreal Engine platform. This article firstly explains the principle of path-finding algorithms that are several commonly used in static environment. Then it compares the performance of different algorithms, and selects the optimal solution among several schemes by summarizing the experimental results. Then based on the optimal path-finding algorithm, using blueprint programming to implement AI path-finding in the Unreal engine. Finally, this article sort out and summarize the overall work process and indicate direction for future optimization.
Contrastive Learning with Hard Negative Entities for Entity Set Expansion

Entity Set Expansion (ESE) is a promising task which aims to expand entities of the target semantic class described by a small seed entity set. Various NLP and IR applications will benefit from ESE due to its ability to discover knowledge. Although previous ESE methods have achieved great progress, most of them still lack the ability to handle hard negative entities (i.e., entities that are difficult to distinguish from the target entities), since two entities may or may not belong to the same semantic class based on different granularity levels we analyze on. To address this challenge, we devise an entity-level masked language model with contrastive learning to refine the representation of entities. In addition, we propose the ProbExpan, a novel probabilistic ESE framework utilizing the entity representation obtained by the aforementioned language model to expand entities. Extensive experiments and detailed analyses on three datasets show that our method outperforms previous state-of-the-art methods.
Supporting Jury Understanding of Expert Evidence in a Virtual Environment

This work investigates the use of Virtual Reality (VR) to present forensic evidence to the jury in a courtroom trial. The findings of a between-participant user study on comprehension of an expert statement are presented, examining the benefits and issues of using VR compared to traditional courtroom presentation (being still images). Participants listened to a forensic scientist explain bloodstain spatter patterns while viewing a mock crime scene in either VR or as still images in video format. Under these conditions, we compared understanding of the expert domain, mental effort and content recall. We found that VR significantly improves the understanding of spatial information and knowledge acquisition. We also identify different patterns of user behaviour depending on the display method. We conclude with suggestions on how to best adapt evidence presentation to VR.
Evaluating Visual Cues for Future Airborne Surveillance Using Simulated Augmented Reality Displays

This work explores the interaction between Augmented Reality (AR) and eye accommodation for airborne surveillance by simulating AR environments in Virtual Reality (VR). We simulate the AR display as displays with the capabilities needed for airborne surveillance are limited and because it would be hazardous to experiment directly on surveillance aircraft. While there is precedent for simulating AR in a VR environment, our study account for two of the physical and physiological aspects of AR: we factor in the focal plane of the AR technology and simulate the eye accommodation reflex of the user to provide focus. We ran a study with 24 participants examining AR cues to support visual search. We also compare the effects of having secondary tasks (that surveillance operators are normally responsible for) directly on the observation window using AR. Our results show that the effectiveness of the AR cues is dependent on the modality of the secondary task. We also found that, under certain situations, operators&#x2019; performances for the search task are improved if the focal plane of the AR display is at the same distance as subsequent search targets.
An Intelligent Framework for Crime Prediction Using Behavioural Tracking and Motion Analysis

Closed Circuit Television Systems are now being deployed in most public spaces to make the city more secure. Manual observation of these clips for the prevention of crime would take up a lot of manpower. This paper proposes an Intelligent Framework using the power of Artificial Intelligence to ensure the safety of the surroundings. The system will use different Computer Vision techniques for video analysis. It will monitor CCTV footage for any criminal offenders, violent objects, and suspicious behavior which could lead to crime. SSD Mobilenet Model, an architecture for concealed object detection, is trained for labeling weapons in the frame. The images captured are processed using Face Detection algorithms to identify human faces. Facial Recognition API using libraries in python is implemented to recognize the offenders from criminal records. A ResNet-GRU Model was trained for human behavior analysis which detects suspicious actions. An alert is generated when there are signs of crime and concerned authorities are notified. The proposed framework aims to make societies secure by correctly identifying criminals and crime-related objects.
Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion

Head pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification.
Hybrid controlled-sum gate with one superconducting qutrit and one cat-state qutrit and application in hybrid entangled state preparation

Compared with a qubit, a qudit (i.e., $d$-level or $d$-state quantum system) provides a larger Hilbert space to store and process information. On the other hand, qudit-based hybrid quantum computing usually requires performing hybrid quantum gates with qudits different in their nature or in their encoding format. In this work, we consider the qutrit case, i.e., the case for a qudit with $d$=3. We propose a simple method to realize a hybrid quantum controlled-SUM gate with one superconducting (SC) qutrit and a cat-state qutrit. This gate plus single-qutrit gates form a universal set of ternary logic gates for quantum computing with qutrits. Our proposal is based on circuit QED and operates essentially by employing a SC ququart (a four-level quantum system) dispersively coupled to a microwave cavity. The gate implementation is quite simple because it only requires a single basic operation. Neither classical pulse nor measurement is needed. The auxiliary higher energy level of the SC ququart is virtually excited during the gate operation, thus decoherence from this level is greatly suppressed. As an application of this gate, we discuss the generation of a hybrid maximally-entangled state of one SC qutrit and one cat-state qutrit. We further analyze the experimental feasibility of creating such hybrid entangled state in circuit QED. This proposal is quite general and can be extended to accomplish the same task in a wide range of physical system, such as a four-level natural or artificial atom coupled to an optical or microwave cavity.
VCD: View-Constraint Disentanglement for Action Recognition

Action recognition is a hot topic in computer vision due to its wide range of applications in urban surveillance. Although some methods are more advanced from an invariant view perspective, those approaches do not perform well for the viewpoint change. To address this issue, one possible solution is tantamount to track the view-invariant representation as it evolves with the performed action. However, the views’ and actions’ performance always complement each other, once simply looking for the view-invariant representation may cause some behavior information to be lost. In this paper, we propose the View-Constraint Disentanglement (VCD) framework for cross-view action recognition. Specifically, Constraint Disentanglement Module (CDM) is utilized to learn an action-invariant representation by discretizing view-specific representation and its normal distribution, which resolves the entangled relationship between view and action. Moreover, a novel Adaptive Distribution Module (ADM) is intended to befit enhance the high-correlation viewpoint variation information and refine the suitable weight. Extensive experiments are conducted on public benchmarks, indicating that our approach achieves better performance than other state-of-the-art approaches.
Cloud of Things in Crowd Engineering: A Tile-Map-Based Method for Intelligent Monitoring of Outdoor Crowd Density

Managing citizen and community safety is one of the most essential services that future cities will require. Crowd analysis and monitoring are also a high priority in the current COVID-19 pandemic scenario, especially because large-scale gatherings can significantly increase the risk of infection transmission. However, crowd tracking presents several complex technical challenges, including accurate people counting and privacy preservation. In this study, using a tile-map-based method, a new intelligent method is proposed which is integrated with the cloud of things and data analytics to provide intelligent monitoring of outdoor crowd density. The proposed system can detect and intelligently analyze the pattern of crowd activity to implement contingency plans, reducing accidents, ensuring public safety, and establishing a smart city. The experimental results demonstrate the feasibility of the proposed model in detecting crowd density status in real-time. It can effectively assist with crowd management tasks such as monitoring, guiding, and managing crowds to ensure safety. In addition, the proposed algorithm provides acceptable performance.
Co-Imagining the Future of Playable Cities

Here we present a co-design exploration into the potential of technology to playfully re-signify urban spaces. We created a speculative catalog of urban tech and used it to facilitate multi-stakeholder discussions about the playful potential of smart cities. The learnings from our co-design engagements embody different people's ideas of how tech might and might not support rich forms of urban play, and contribute to ongoing efforts at exploring how to playfully reconfigure our cities. We present: (1) a list of inspirational play potentials of urban spaces—i.e. playful things already people do, and enjoy, in the public space; (2) a portfolio of speculative ideas that show how tech might help to realize that potential; and (3) a discussion of stakeholders' responses to these ideas. Our work can provide designers with inspiration and actionable advice for cultivating forms of urban play that cater to people's socio-emotional needs.
Purity of an ST monad: full abstraction by semantically typed back-translation

In 1995, Launchbury and Peyton Jones extended Haskell with an ST monad that allows the programmer to use higher-order mutable state. They informally argued that these state computations were safely encapsulated, and as such, that the rich reasoning principles stemming from the purity of the language, were not threatened. In this paper, we give a formal account of the preservation of purity after adding an ST monad to a simply-typed call-by-value recursive lambda calculus. We state and prove full abstraction when embedding the pure language into its extension with ST; contextual equivalences from the pure language continue to hold in the presence of ST. Proving full abstraction of compilers is usually done by emulating or back-translating the target features (here: ST computations) into the source language, a well-known challenge in the secure compilation community. We employ a novel proof technique for proving our full abstraction result that allows us to use a semantically (but not syntactically) typed back-translation into an intermediate language. We believe that this technique provides additional insight into our proof and that it is of general interest to researchers studying programming languages and compilers using full abstraction. The results presented here are fully formalized in the Coq proof assistant using the Iris framework.
An Effective Workflow Scheduling Algorithm in Cloud Computing Using Cat Swarm Optimization

Effective workflow scheduling approach is needed, as it is huge challenge in cloud computing as variable and heterogeneous workflows comes onto cloud console. To handle these complex workflows and to schedule them onto appropriate virtual resources an effective workflow scheduler is necessary. Earlier authors used various nature inspired algorithms to solve this workflow scheduling by minimizing makespan and maximizing resource utilization, but still there is a chance to minimize makespan and improve resource utilization. In this paper, we have calculated the task priorities of all tasks, which are incoming onto cloud console, and thereby mapping tasks onto VMs to effectively map tasks. Cat swarm optimization used to solve scheduling problem and finally we have addressed parameters named as makespan and resource utilization. Simulation was carried out on workflowsim and we evaluated the efficacy of our algorithm by comparing with existing algorithms PSO and CS. From simulation results we observed that our algorithm shows great improvement over existing algorithms for mentioned parameters.
COSPLAY

Maintaining a consistent persona is essential for building a human-like conversational model. However, the lack of attention to the partner makes the model more egocentric: they tend to show their persona by all means such as twisting the topic stiffly, pulling the conversation to their own interests regardless, and rambling their persona with little curiosity to the partner. In this work, we propose COSPLAY(COncept Set guided PersonaLized dialogue generation Across both partY personas) that considers both parties as a "team": expressing self-persona while keeping curiosity toward the partner, leading responses around mutual personas, and finding the common ground. Specifically, we first represent self-persona, partner persona and mutual dialogue all in the concept sets. Then, we propose the Concept Set framework with a suite of knowledge-enhanced operations to process them such as set algebras, set expansion, and set distance. Based on these operations as medium, we train the model by utilizing 1) concepts of both party personas, 2) concept relationship between them, and 3) their relationship to the future dialogue. Extensive experiments on a large public dataset, Persona-Chat, demonstrate that our model outperforms state-of-the-art baselines for generating less egocentric, more human-like, and higher quality responses in both automatic and human evaluations.