Reviewerly/exHarmony huggingface dataset:
PolarMOT: How Far Can Geometric Relations Take us in 3D Multi-object Tracking?

Most (3D) multi-object tracking methods rely on appearance-based cues for data association. By contrast, we investigate how far we can get by only encoding geometric relationships between objects in 3D space as cues for data-driven data association. We encode 3D detections as nodes in a graph, where spatial and temporal pairwise relations among objects are encoded via localized polar coordinates on graph edges. This representation makes our geometric relations invariant to global transformations and smooth trajectory changes, especially under non-holonomic motion. This allows our graph neural network to learn to effectively encode temporal and spatial interactions and fully leverage contextual and motion cues to obtain final scene interpretation by posing data association as edge classification. We establish a new state-of-the-art on nuScenes dataset and, more importantly, show that our method, PolarMOT, generalizes remarkably well across different locations (Boston, Singapore, Karlsruhe) and datasets (nuScenes and KITTI).
AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture

To address the ill-posed problem caused by partial observations in monocular human volumetric capture, we present AvatarCap, a novel framework that introduces animatable avatars into the capture pipeline for high-fidelity reconstruction in both visible and invisible regions. Our method firstly creates an animatable avatar for the subject from a small number ( $$\sim $$ 20) of 3D scans as a prior. Then given a monocular RGB video of this subject, our method integrates information from both the image observation and the avatar prior, and accordingly reconstructs high-fidelity 3D textured models with dynamic details regardless of the visibility. To learn an effective avatar for volumetric capture from only few samples, we propose GeoTexAvatar, which leverages both geometry and texture supervisions to constrain the pose-dependent dynamics in a decomposed implicit manner. An avatar-conditioned volumetric capture method that involves a canonical normal fusion and a reconstruction network is further proposed to integrate both image observations and avatar dynamics for high-fidelity reconstruction in both observed and invisible regions. Overall, our method enables monocular human volumetric capture with detailed and pose-dependent dynamics, and the experiments show that our method outperforms state of the art.
Performance of surface codes in realistic quantum hardware

Surface codes are generally studied based on the assumption that each of the qubits that make up the surface code lattice suffers noise that is independent and identically distributed (i.i.d.). However, real benchmarks of the individual relaxation ($T_1$) and dephasing ($T_2$) times of the constituent qubits of state-of-the-art quantum processors have recently shown that the decoherence effects suffered by each particular qubit actually vary in intensity. In consequence, in this article we introduce the independent non-identically distributed (i.ni.d.) noise model, a decoherence model that accounts for the non-uniform behaviour of the docoherence parameters of qubits. Additionally, we use the i.ni.d model to study how it affects the performance of a specific family of Quantum Error Correction (QEC) codes known as planar codes. For this purpose we employ data from four state-of-the-art superconducting processors: ibmq\_brooklyn, ibm\_washington, Zuchongzhi and Rigetti Aspen-M-1. Our results show that the i.i.d. noise assumption overestimates the performance of surface codes, which can suffer up to $95\%$ performance decrements in terms of the code pseudo-threshold when they are subjected to the i.ni.d. noise model. Furthermore, we consider and describe two methods which enhance the performance of planar codes under i.ni.d. noise. The first method involves a so-called re-weighting process of the conventional minimum weight perfect matching (MWPM) decoder, while the second one exploits the relationship that exists between code performance and qubit arrangement in the surface code lattice. The optimum qubit configuration derived through the combination of the previous two methods can yield planar code pseudo-threshold values that are up to $650\%$ higher than for the traditional MWPM decoder under i.ni.d. noise.
F-BIDS: Federated-Blending based Intrusion Detection System

The rapid development of network communication along with the drastic increase in the number of smart devices has triggered a surge in network traffic, which can contain private data and in turn affect user privacy. Recently, Federated Learning (FL) has been proposed in Intrusion Detection Systems (IDS) to ensure attack detection, privacy preservation, and cost reduction, which are crucial issues in traditional centralized machine-learning-based IDS. However, FL-based approaches still exhibit vulnerabilities that can be exploited by adversaries to compromise user data. At the same time, meta-models (including the blending models) have been recognized as one of the solutions to improve generalization for attack detection and classification since they enhance generalization and predictive performances by combining multiple base models. Therefore, in this paper, we propose a Federated Blending model-driven IDS framework for the Internet of Things (IoT) and Industrial IoT (IIoT), called F-BIDS, in order to further protect the privacy of existing ML-based IDS. The proposition consists of a Decision Tree (DT) and Random Forest (RF) as base classifiers to first produce the meta-data. Then, the meta-classifier, which is a Neural Networks (NN) model, uses the meta-data during the federated training step, and finally, it makes the final classification on the test set. Specifically, in contrast to the classical FL approaches, the federated meta-classifier is trained on the meta-data (composite data) instead of user-sensitive data to further enhance privacy. To evaluate the performance of F-BIDS, we used the most recent and open cyber-security datasets, called Edge-IIoTset (published in 2022) and InSDN (in 2020). We chose these datasets because they are recent datasets and contain a large amount of network traffic including both malicious and benign traffic.
Determining the fullness of garbage containers by deep learning

An essential point in waste management, which is a matter of great importance for the environment and nature, is waste collection from temporary storage points. Since the garbage collection process is generally time-related, sometimes the garbage containers overflow or empty. Intelligent services are being developed for issues related to the cleanliness of the streets through cameras and specially designed monitoring tools. This study has investigated whether deep learning can determine if the garbage containers are full or not based on the camera images. For this purpose, experiments were carried out for automatic classification processes by applying DenseNet-169, EfficientNet-B3, MobileNetV3-Large, and VGG19-Bn deep learning algorithms on the CDCM dataset, which contains images of trash cans or containers labeled as clean and dirty. With a 94.931% accuracy rate, it has been found that an intelligent system can be used successfully in smart cities to determine the status of garbage and garbage containers on the streets and inform the authorities.
Federated Meta-Learning with Attention for Diversity-Aware Human Activity Recognition

The ubiquity of smartphones equipped with multiple sensors has provided the possibility of automatically recognizing of human activity, which can benefit intelligent applications such as smart homes, health monitoring, and aging care. However, there are two major barriers to deploying an activity recognition model in real-world scenarios. Firstly, deep learning models for activity recognition use a large amount of sensor data, which are privacy-sensitive and hence cannot be shared or uploaded to a centralized server. Secondly, divergence in the distribution of sensory data exists among multiple individuals due to their diverse behavioral patterns and lifestyles, which contributes to difficulty in recognizing activity for large-scale users or ’cold-starts’ for new users. To address these problems, we propose DivAR, a diversity-aware activity recognition framework based on a federated Meta-Learning architecture, which can extract general sensory features shared among individuals by a centralized embedding network and individual-specific features by attention module in each decentralized network. Specifically, we first classify individuals into multiple clusters according to their behavioral patterns and social factors. We then apply meta-learning in the architecture of federated learning, where a centralized meta-model learns common feature representations that can be transferred across all clusters of individuals, and multiple decentralized cluster-specific models are utilized to learn cluster-specific features. For each cluster-specific model, a CNN-based attention module learns cluster-specific features from the global model. In this way, by training with sensory data locally, privacy-sensitive information existing in sensory data can be preserved. To evaluate the model, we conduct two data collection experiments by collecting sensor readings from naturally used smartphones annotated with activity information in the real-life environment and constructing two multi-individual heterogeneous datasets. In addition, social characteristics including personality, mental health state, and behavior patterns are surveyed using questionnaires. Finally, extensive empirical results demonstrate that the proposed diversity-aware activity recognition model has a relatively better generalization ability and achieves competitive performance on multi-individual activity recognition tasks.
Bayesian Network–Based Fault Diagnostic System for Nuclear Power Plant Assets

Future advances in nuclear power technologies call for enhanced operator advice and autonomous control capabilities that can leverage simpler designs and increased safety features to reduce reliance on human labor. One of the first tasks in the development of such capabilities is the formulation of symptom-based conditional failure probabilities for the plant structures, systems, and components (SSCs) of interest. The primary goal is to aid plant personnel in (1) deducing the probabilistic performance status of the monitored SSCs and (2) detecting impending faults/failures. The task of estimating conditional failure probability is a bidirectional inference problem, and a logical approach is to use the Bayesian network (BN) method. As a knowledge-based explainable artificial intelligence tool and a probabilistic graphical model, BN offers reasoning capability under uncertainty, graphical representation emulating physical behavior of the target SSC, and interpretability of the model structure and results. This paper provides a systematic overview of the BN technique and the software tools for implementing BN models, along with the associated knowledge representation and reasoning paradigm. Both operational data and expert judgment can be readily incorporated into the knowledge base of a BN model. The challenges with data availability are highlighted, and the general approach to target SSC identification is presented. The focus is on failure-prone and risk-important balance of plant assets, especially for cases with strong operator involvement. Two example case studies on the failure of (1) a centrifugal pump and (2) an electric motor are conducted to demonstrate the usefulness and technical feasibility of the proposed BN-based fault diagnostic system using an expert system shell.
Computer Stereo Vision for Autonomous Driving: Theory and Algorithms

As an important component of autonomous systems, autonomous car perception has had a big leap with recent advances in parallel computing architectures. With the use of tiny but full-feature embedded supercomputers, computer stereo vision has been prevalently applied in autonomous cars for depth perception. The two key aspects of computer stereo vision are speed and accuracy. They are both desirable but conflicting properties, as the algorithms with better disparity accuracy usually have higher computational complexity. Therefore, the main aim of developing a computer stereo vision algorithm for resource-limited hardware is to improve the trade-off between speed and accuracy. In this chapter, we introduce both the hardware and software aspects of computer stereo vision for autonomous car systems. Then, we discuss four autonomous car perception tasks, including (1) visual feature detection, description and matching, (2) 3D information acquisition, (3) object detection/recognition and (4) semantic image segmentation. The principles of computer stereo vision and parallel computing on multi-threading CPU and GPU architectures are then detailed.
Optimal Pricing Strategy for Data Center Considering Demand Response and Renewable Energy Source Accommodation

With the continuous development of information technology, data centers (DCs) consume significant and ever-growing amounts of electrical energy. Renewable energy sources (RESs) can act as clean solutions to meet this requirement without polluting the environment. Each DC serves numerous users for their data service demands, which are regarded as flexible loads. In this paper, the willingness to pay and time sensitivities of DC users are firstly explored, and the user-side demand response is then devised to improve the overall benefits of DC operation. Then, a Stackelberg game between a DC and its users is proposed. The upper-level model aims to maximize the profit of the DC, in which the time-varying pricing of data services is optimized, and the lower-level model addresses user's optimal decisions for using data services while balancing their time and cost requirements. The original bi-level optimization problem is then transformed into a single-level problem using the Karush-Kuhn-Tucker optimality conditions and strong duality theory, which enables the problem to be solved efficiently. Finally, case studies are conducted to demonstrate the feasibility and effectiveness of the proposed method, as well as the effects of the time-varying data service price mechanism on the RES accommodation.
HEDA

Recent years have witnessed the rapid development of the encrypted database, due to the increasing number of data privacy breaches and the corresponding laws and regulations that caused millions of dollars in loss. These encrypted databases may rely on different techniques, such as cryptographic primitives and trusted execution environments. In this work, we investigate the feasibility of utilizing fully homomorphic encryption (FHE) to support unbounded database aggregation queries, which typically involve comparisons as filtering predicates and a final aggregation. These operators are theoretically supported by FHE, but need careful algorithm design to maximize the efficiency and have not been explored before. We creatively use two types of FHE schemes, i.e. , one for numerical and one for binary value, to enjoy their advantages respectively. To bridge the encrypted values between these two schemes for seamless query processing without client-server interaction, we propose a novel ciphertext transformation mechanism, which is of independent research interest, to close this gap. We further implement our system and test it over three TPC-H queries and a query over a real social media e-commerce database. Evaluation results show that, to process an aggregation query over 8 k encrypted rows takes about 430 seconds. Although it is slower than plaintext processing in magnitudes and still has much room for improvement, as the very first work in this domain, our system demonstrates the feasibility of using FHE to process OLAP queries.
Listen2YourHeart: A Self-Supervised Approach for Detecting Murmur in Heart-Beat Sounds

Heart murmurs are abnormal sounds present in heartbeats, caused by turbulent blood flow through the heart.The PhysioNet 2022 challenge targets automatic detection of murmur from audio recordings of the heart and automatic detection of normal vs. abnormal clinical outcome.The recordings are captured from multiple locations around the heart.Our participation investigates the effectiveness of selfsupervised learning for murmur detection.We train the layers of a backbone CNN in a self-supervised way with data from both this year's and the 2016 challenge.We use two different augmentations on each training sample, and normalized temperature-scaled cross-entropy loss.We experiment with different augmentations to learn effective phonocardiogram representations.To build the final detectors we train two classification heads, one for each challenge task.We present evaluation results for all combinations of the available augmentations, and for our multipleaugmentation approach.Our team's, Listen2YourHeart, SSL murmur detection classifier received a weighted accuracy score of 0.737 (ranked 13th out of 40 teams) and an outcome identification challenge cost score of 11946 (ranked 7th out of 39 teams) on the hidden test set.
Spatiotemporal variation of vegetation cover in mining areas of Dexing City, China

Dexing City is an important mining city in China, abounding in copper ore, lead ore, zinc ore, and other metal resources, and there are two large open-pit mines in its territory, Dexing Copper Mine and Yinshan Mine. The two open-pit mines have been expanding their mining production scale since 2005, with frequent mining activities; and the expansion of the pits and the discharge of solid waste will undoubtedly increase the land use and cause the destruction of vegetation. Therefore, we plan to visualize the change in vegetation cover in Dexing City from 2005 to 2020 and the expansion of the two open-pit mines by calculating changes of the Fractional Vegetation Cover (FVC) in the mining area using remote sensing technology. In this study, we calculated the FVC of Dexing City in 2005, 2010, 2015 and 2020 using data from NASA Landsat Database via ENVI image analysis software, plotted the FVC reclassified maps via ArcGIS, and conducted field investigations in the mining areas of Dexing City. In this way, we can visualize the spatial and temporal changes of vegetation cover in Dexing City from 2005 to 2020, and appreciate the situation of mining expansion and its solid waste discharge in Dexing City. The results of this study showed that the vegetation cover of Dexing City remained stable from 2005 to 2020, as the expansion of mining scale and mine pits was accompanied by active environmental management and land reclamation, setting a positive example for other mining cities.
Multicontrast MRI Super-Resolution via Transformer-Empowered Multiscale Contextual Matching and Aggregation

Magnetic resonance imaging (MRI) possesses the unique versatility to acquire images under a diverse array of distinct tissue contrasts, which makes multicontrast super-resolution (SR) techniques possible and needful. Compared with single-contrast MRI SR, multicontrast SR is expected to produce higher quality images by exploiting a variety of complementary information embedded in different imaging contrasts. However, existing approaches still have two shortcomings: 1) most of them are convolution-based methods and, hence, weak in capturing long-range dependencies, which are essential for MR images with complicated anatomical patterns and 2) they ignore to make full use of the multicontrast features at different scales and lack effective modules to match and aggregate these features for faithful SR. To address these issues, we develop a novel multicontrast MRI SR network via transformer-empowered multiscale feature matching and aggregation, dubbed McMRSR <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{++}$</tex-math> </inline-formula> . First, we tame transformers to model long-range dependencies in both reference and target images at different scales. Then, a novel multiscale feature matching and aggregation method is proposed to transfer corresponding contexts from reference features at different scales to the target features and interactively aggregate them Furthermore, a texture-preserving branch and a contrastive constraint are incorporated into our framework for enhancing the textural details in the SR images. Experimental results on both public and clinical in vivo datasets show that McMRSR <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{++}$</tex-math> </inline-formula> outperforms state-of-the-art methods under peak signal to noise ratio (PSNR), structure similarity index measure (SSIM), and root mean square error (RMSE) metrics significantly. Visual results demonstrate the superiority of our method in restoring structures, demonstrating its great potential to improve scan efficiency in clinical practice.
Voice User Interface: Literature review, challenges and future directions

Natural user interfaces are increasingly popular these days. One of the most common of these user interfaces today are voice-activated interfaces, in particular intelligent voice assistants such as Google Assistant, Alexa, Cortana and Siri. However, the results show that although there are many services available, there is still a lot to be done to improve the usability of these systems. Speech recognition, contextual understanding and human interaction are the issues that are not yet solved in this field. In this context, this research paper focuses on the state of the art and knowledge of work on intelligent voice interfaces, challenges and issues related to this field, in particular on interaction quality, usability, security and usability. As such, the study also examines voice assistant architecture components following the expansion of the use of technologies such as wearable computing in order to improve the user experience. Moreover, the presentation of new emerging technologies in this field will be the subject of a section in this work. The main contributions of this paper are therefore: (1) overview of existing research, (2) analysis and exploration of the field of intelligent voice assistant systems, with details at the component level, (3) identification of areas that require further research and development, with the aim of increasing its use, (4) various proposals for research directions and orientations for future work, and finally, (5) study of the feasibility of designing a new type of voice assistant and general presentation of the latter, whose realisation will be the subject of a thesis.