Reviewerly/exHarmony huggingface dataset:
Hi4D: 4D Instance Segmentation of Close Human Interaction

We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11 K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks. Hi4D dataset can be found at https://ait.ethz.ch/Hi4D.
Semi-DETR: Semi-Supervised Object Detection with Detection Transformers

We analyze the DETR-based framework on semi-supervised object detection (SSOD) and observe that (1) the one-to-one assignment strategy generates incorrect matching when the pseudo ground-truth bounding box is inaccurate, leading to training inefficiency; (2) DETR-based detectors lack deterministic correspondence between the input query and its prediction output, which hinders the applicability of the consistency-based regularization widely used in current SSOD methods. We present Semi-DETR, the first transformer-based end-to-end semi-supervised object detector, to tackle these problems. Specifically, we propose a Stage-wise Hybrid Matching strategy that combines the one-to-many assignment and one-to-one assignment strategies to improve the training efficiency of the first stage and thus provide high-quality pseudo labels for the training of the second stage. Besides, we introduce a Cross-view Query Consistency method to learn the semantic feature invariance of object queries from different views while avoiding the need to find deterministic query correspondence. Furthermore, we propose a Cost-based Pseudo Label Mining module to dynamically mine more pseudo boxes based on the matching cost of pseudo ground truth bounding boxes for consistency training. Extensive experiments on all SSOD settings of both COCO and Pascal VOC benchmark datasets show that our Semi-DETR method outperforms all state-of-the-art methods by clear margins.
VideoTrack: Learning to Track Objects via Video Transformer

Existing Siamese tracking methods, which are built on pair-wise matching between two single frames, heavily rely on additional sophisticated mechanism to exploit temporal information among successive video frames, hindering them from efficiency and industrial deployments. In this work, we resort to sequence-level target matching that can encode temporal contexts into the spatial features through a neat feedforward video model. Specifically, we adapt the standard video transformer architecture to visual tracking by enabling spatiotemporal feature learning directly from frame-level patch sequences. To better adapt to the tracking task, we carefully blend the spatiotemporal information in the video clips through sequential multi-branch triplet blocks, which formulates a video transformer backbone. Our experimental study compares different model variants, such as tokenization strategies, hierarchical structures, and video attention schemes. Then, we propose a disentangled dual-template mechanism that decouples static and dynamic appearance clues over time, and reduces temporal redundancy in video frames. Extensive experiments show that our method, named as Video Track, achieves state-of-the-art results while running in real-time.
Backdoor Attacks Against Deep Image Compression via Adaptive Frequency Trigger

Recent deep-learning-based compression methods have achieved superior performance compared with traditional approaches. However, deep learning models have proven to be vulnerable to backdoor attacks, where some specific trigger patterns added to the input can lead to malicious behavior of the models. In this paper, we present a novel backdoor attack with multiple triggers against learned image compression models. Motivated by the widely used discrete cosine transform (DCT) in existing compression systems and standards, we propose a frequency-based trigger injection model that adds triggers in the DCT domain. In particular, we design several attack objectives for various attacking scenarios, including: 1) attacking compression quality in terms of bit-rate and reconstruction quality; 2) attacking task-driven measures, such as downstream face recognition and semantic segmentation. Moreover, a novel simple dynamic loss is designed to balance the influence of different loss terms adaptively, which helps achieve more efficient training. Extensive experiments show that with our trained trigger injection models and simple modification of encoder parameters (of the compression model), the proposed attack can successfully inject several backdoors with corresponding triggers in a single image compression model.
Leading Virtual Reality (VR) and Augmented Reality (AR) in Education: Bibliometric and Content Analysis From the Web of Science (2018–2022)

Emerging virtual reality and augmented reality (VR/AR) technologies in education have the potential to revolutionize the whole educational experience. Studies have shown that before introducing the technology to their pupils, educators should do a thorough assessment of the technology’s viability and possible acceptance in the classroom. However, not much is synthesized from the body of knowledge about the leadership and coupling of VR and AR in education, including its trends and direction of research in other domains. In this article, the bibliometric and content analysis has shown that interest in virtual reality and augmented reality has increased from 2018 to 2022 in the Web of Science database. The patterns formed from Vosviewer in the bibliometric analysis have also portrayed that Virtual reality (VR) and Augmented Reality (AR) are used in many fields, including education, medicine, arts, engineering, business, and marketing. Other results from data mining showed the authors with the most publications in the field, their affiliated universities, and the source of publications. In the wake of the Covid-19 epidemic, these findings may also help improve educational opportunities for all students and the quality of instruction they receive. As implications, Virtual reality (VR) and augmented reality (AR) harness current technology to put digital content in the actual world, enabling consumers to have a more meaningful interaction with it on desktop or hand-held devices. This bibliometric research shows that animation, 3D graphics, and sound may replace more traditional ways of schooling in the future.
A Survey of Video Surveillance Systems in Smart City

Smart cities are being developed worldwide with the use of technology to improve the quality of life of citizens and enhance their safety. Video surveillance is a key component of smart city infrastructure, as it involves the installation of cameras at strategic locations throughout the city for monitoring public spaces and providing real-time surveillance footage to law enforcement and other city representatives. Video surveillance systems have evolved rapidly in recent years, and are now integrated with advanced technologies like deep learning, blockchain, edge computing, and cloud computing. This study provides a comprehensive overview of video surveillance systems in smart cities, as well as the functions and challenges of those systems. The aim of this paper is to highlight the importance of video surveillance systems in smart cities and to provide insights into how they could be used to enhance safety, security, and the overall quality of life for citizens.
DDH-QA: A Dynamic Digital Humans Quality Assessment Database

In recent years, large amounts of effort have been put into pushing forward the real-world application of dynamic digital human (DDH). However, most current quality assessment research focuses on evaluating static 3D models and usually ignores motion distortions. Therefore, in this paper, we construct a large-scale dynamic digital human quality assessment (DDH-QA) database with diverse motion content as well as multiple distortions to comprehensively study the perceptual quality of DDHs. Both model-based distortion (noise, compression) and motion-based distortion (binding error, motion unnaturalness) are taken into consideration. Ten types of common motion are employed to drive the DDHs and a total of 800 DDHs are generated in the end. Afterward, we render the video sequences of the distorted DDHs as the evaluation media and carry out a well-controlled subjective experiment. Then a benchmark experiment is conducted with the state-of-the-art video quality assessment (VQA) methods and the experimental results show that existing VQA methods are limited in assessing the perceptual loss of DDHs. The database is available at https://github.com/zzc-1998/DDH-QA.
LKD-Net: Large Kernel Convolution Network for Single Image Dehazing

The previous deep CNN-based single-image dehazing methods are devoted to improving the performance by increasing the network’s depth and width. In this paper, a novel Large Kernel Convolution Dehaze Network (LKD-Net) is proposed to enhance the network’s performance by increasing the size of the convolutional kernel. The main module in LKD-Net is the designed Large Kernel Convolution Dehaze Block (LKD Block), which consists of the Decomposition deep-wise Large Kernel Convolution Block (DLKCB) and the Channel Enhanced Feed-forward Network (CEFN). DLKCB is designed to reduce the massive amount of computational overhead and parameters of the large kernel by splitting the deep-wise large kernel convolution into a smaller depth-wise convolution and a depth-wise dilated convolution. Meanwhile, CEFN is designed to enhance the robustness of the Feed-forward Network by exploiting significant channels. The extensive experiments on RESIDE dataset demonstrate that our LKD-Net outperforms the state-of-the-art with far less computational overhead and parameters.
Deep reinforcement learning assisted co-evolutionary differential evolution for constrained optimization

Solving constrained optimization problems (COPs) with evolutionary algorithms (EAs) is a popular research direction due to its potential and diverse applications. One of the key issues in solving COPs is the choice of constraint handling techniques (CHTs), as different CHTs can lead to different evolutionary directions. Combining EAs with deep reinforcement learning (DRL) is a promising and emerging approach for solving COPs. Although DRL can help solve the problem of pre-setting operators in EAs, neural networks need to obtain diverse training data within a limited number of evaluations in EAs. Based on the above considerations, this work proposes a DRL assisted co-evolutionary differential evolution, named CEDE-DRL, which can effectively use DRL to help EAs solve COPs. (1) This method incorporates co-evolution into the extraction of training data for the first time, ensuring the diversity of samples and improving the accuracy of the neural network model through information exchange between multiple populations. (2) Multiple CHTs are used for offspring selection to ensure the algorithm's generality and flexibility. (3) DRL is used to evaluate the population state, taking into account feasibility, convergence, and diversity in the state setting and using the overall degree of improvement as a reward. The neural network selects suitable parent populations and corresponding archives for mutation. Finally, (4) to avoid premature convergence and local optima, an adaptive operator selection and individual archive elimination mechanism is added. Comparisons with state-of-the-art algorithms on benchmark functions CEC2010 and CEC2017 show that the proposed method performs competitively and produced robust solutions. The results of the application test set CEC2020 show that the proposed algorithm is also effective in real-world problems.
Artificial Neural Network Chaotic PRNG and simple encryption on FPGA

Artificial Neural Networks (ANNs) are remarkably able to fit complex functions, making them useful in various applications and systems. This paper uses ANN to fit the Pehlivan–Uyaroglu Chaotic System (PUCS) to produce an Artificial Neural Network Chaotic Pseudo-Random Number Generator (ANNC-PRNG). The proposed PRNG imitates the PUCS chaotic system's properties and attractor shape. The proposed ANNC-PRNG is implemented in a simple image encryption system on the Xilinx Kintex-7 Genesys 2 Field Programmable Gate Array (FPGA) board. Hardware realization of an ANN trained on chaotic time series has not been presented before. The proposed ANN can be used for different numerical methods or chaotic systems, including fractional-order systems while keeping the same resources despite the methods' complexity or chaotic systems' complexity. Extensive testing for the ANNC-PRNG was done to prove the randomness of the produced outputs. The proposed ANNC-PRNG and the encryption system passed various well-established security and statistical tests and produced good results compared to recent similar research. The encryption system is robust against different attacks. The proposed hardware architecture is fast as it reaches a maximum frequency of 12.553 MHz throughput of 301 Mbit/s.
Foreign Object Debris Detection for Airport Pavement Images Based on Self-Supervised Localization and Vision Transformer

Supervised object detection methods provide subpar performance when applied to Foreign Object Debris (FOD) detection because FOD could be arbitrary objects according to the Federal Aviation Administration (FAA) specification. Current supervised object detection algorithms require datasets that contain annotated examples of every to-be-detected object. While a large and expensive dataset could be developed to include common FOD examples, it is infeasible to collect all possible FOD examples in the dataset representation because of the open-ended nature of FOD. Limitations of the dataset could cause FOD detection systems driven by those supervised algorithms to miss certain FOD, which can become dangerous to airport operations. To this end, this paper presents a self-supervised FOD localization by learning to predict the runway images, which avoids the enumeration of FOD annotation examples. The localization method utilizes the Vision Transformer (ViT) to improve localization performance. The experiments show that the method successfully detects arbitrary FOD in real-world runway situations. The paper also provides an extension to the localization result to perform classification; a feature that can be useful to downstream tasks. To train the localization, this paper also presents a simple and realistic dataset creation framework that only collects clean runway images. The training and testing data for this method are collected at a local airport using unmanned aircraft systems (UAS). Additionally, the developed dataset is provided for public use and further studies.
A novel approach for assessing rooftop-and-facade solar photovoltaic potential in rural areas using three-dimensional (3D) building models constructed with GIS

Characterization of solar photovoltaic (PV) potential is crucial for promoting renewable energy in rural areas, where there are a large number of roofs and facades ideal for PV module installation. However, accurately estimating solar PV potential on three-dimensional (3D) rural surfaces has been challenging due to the lack of 3D building models. To address this issue, we proposed a novel approach, which for the first time constructs rural 3D building models from publicly available satellite images and vector maps. Based on these models, it precisely evaluates the solar PV potential of rural rooftops and facades. The approach was validated against two realistic 3D village models and on-site solar radiation measurements. Using the validated approach, case studies in a village and on a large-scale island were conducted, respectively. The results showed that rural rooftops facing south and north, and facades facing south and west, have the highest PV potential ranks. North-facing rooftops with a slope of 30° represent 32.7% of the total rooftop solar PV potential, therefore, they should not be neglected in future evaluations. The proposed approach is cost-effective and valid for accurately assessing micro- and macro-scale rural solar PV potential that can facilitate rural renewable energy penetration.
Reconfigurable reservoir computing in a magnetic metamaterial

Abstract In-materia reservoir computing (RC) leverages the intrinsic physical responses of functional materials to perform complex computational tasks. Magnetic metamaterials are exciting candidates for RC due to their huge state space, nonlinear emergent dynamics, and non-volatile memory. However, to be suitable for a broad range of tasks, the material system is required to exhibit a broad range of properties, and isolating these behaviours experimentally can often prove difficult. By using an electrically accessible device consisting of an array of interconnected magnetic nanorings- a system shown to exhibit complex emergent dynamics- here we show how reconfiguring the reservoir architecture allows exploitation of different aspects the system’s dynamical behaviours. This is evidenced through state-of-the-art performance in diverse benchmark tasks with very different computational requirements, highlighting the additional computational configurability that can be obtained by altering the input/output architecture around the material system.
Adaptive differential evolution algorithm based on deeply-informed mutation strategy and restart mechanism

Differential evolution is one of the most powerful stochastic real-parameter optimization algorithms currently, and its performance depends heavily on control parameters and mutation strategy. In recent years, methods to select favorable parameters control and mutation strategy when solving various optimization problems have attracted increasing attention. To choose an appropriate mutation strategy and control parameters for a given optimization problem, in this paper, a Adaptive Differential Evolution Algorithm Based on Deeply-Informed Mutation Strategy and Restart Mechanism (ADEDMR) is proposed, and the ADEDMR algorithm has the following characteristics: First, a deeply-informed mutation strategy is proposed, which takes into account the information of suboptimal solutions discarded by selection and inherits the advantages of the powerful “DE/target-pbest/1/bin”, aiming to obtain a better perceptual landscape of the target function and improve the candidate diversity of the trial vector. Second, according to the evolution process, the segmentation method is used to control F, which alleviates the scaling of F in the wrong direction, and makes the newly generated F fit more accurately. Third, a new population restart mechanism is adopted to further enhance population diversity by adaptively enhancing the search ability of hopeless individuals and randomly replacing some inferior individuals with wavelet walks. To evaluate the performance of our proposed algorithm, comparative experiments are conducted on 72 benchmark functions from the CEC2014, CEC2017 and CEC2022 test suites. Experimental results show that the proposed ADEDMR has higher convergence accuracy, better optimization ability when solving high-dimensional complex functions, and is competitive with six recent strong DE variants.
Generation of two-giant-atom entanglement in waveguide-QED systems

We study the generation of quantum entanglement between two giant atoms coupled to a one-dimensional waveguide. Since each giant atom interacts with the waveguide at two separate coupling points, there exist three different coupling configurations in the two-atom waveguide system: separated, braided, and nested couplings. Within the Wigner-Weisskopf framework for single coupling points, the quantum master equations governing the evolution of the two giant atoms are obtained. For each coupling configuration, the entanglement dynamics of the two giant atoms is studied, including the cases of two different atomic initial states: single- and double-excitation states. It is shown that the generated entanglement depends on the coupling configuration, phase shift, and atomic initial state. For the single-excitation initial state, there exists steady-state entanglement for these three couplings due to the appearance of the dark state. For the double-excitation initial state, an entanglement sudden birth is observed via adjusting the phase shift. In particular, the maximal entanglement for the nested coupling is about one order of magnitude larger than those of separate and braided couplings. In addition, the influence of the atomic frequency detuning on the entanglement generation is studied. This work can be utilized for the generation and control of atomic entanglement in quantum networks based on giant-atom waveguide-QED systems, which have wide potential applications in quantum information processing.
Co-jump dynamicity in the cryptocurrency market: A network modelling perspective

We examine the co-jumps of 37 cryptocurrencies based on a network model, and analyse the portfolio implications. The results reveal that, firstly, Bitcoin exerts the strongest influence. Secondly, co-jump heterogeneity exists across pairs of cryptocurrencies with different market-capitalizations, and the impact of co-jumps is time-varying. Thirdly, the dynamic ranking of co-jump influence shows that, during the COVID-19 pandemic, Bitcoin dominates in the centrality ranking. However, smaller cryptocurrencies (Dogecoin and TRON) exhibit significant co-jump influence. Fourthly, the portfolios constructed based on the co-jump network outperform the baseline strategy by attaining higher returns while experiencing less volatility and shorter downside periods.