Reviewerly/exHarmony huggingface dataset:
fakeWeather: Adversarial Attacks for Deep Neural Networks Emulating Weather Conditions on the Camera Lens of Autonomous Systems

Recently, Deep Neural Networks (DNNs) have achieved remarkable performances in many applications, while several studies have enhanced their vulnerabilities to malicious attacks. In this paper, we emulate the effects of natural weather conditions to introduce plausible perturbations that mislead the DNNs. By observing the effects of such atmospheric perturbations on the camera lenses, we model the patterns to create different masks that fake the effects of rain, snow, and hail. Even though the perturbations introduced by our attacks are visible, their presence remains unnoticed due to their association with natural events, which can be especially catastrophic for fully-autonomous and unmanned vehicles. We test our proposed fake Weather attacks on multiple Convolutional Neural Network and Capsule Network models, and report noticeable accuracy drops in the presence of such adversarial perturbations. Our work introduces a new security threat for DNNs, which is especially severe for safety-critical applications and autonomous systems.
D &amp;D: Learning Human Dynamics from Dynamic Camera

3D human pose estimation from a monocular video has recently seen significant improvements. However, most state-of-the-art methods are kinematics-based, which are prone to physically implausible motions with pronounced artifacts. Current dynamics-based methods can predict physically plausible motion but are restricted to simple scenarios with static camera view. In this work, we present D &D (Learning Human Dynamics from Dynamic Camera), which leverages the laws of physics to reconstruct 3D human motion from the in-the-wild videos with a moving camera. D &D introduces inertial force control (IFC) to explain the 3D human motion in the non-inertial local frame by considering the inertial forces of the dynamic camera. To learn the ground contact with limited annotations, we develop probabilistic contact torque (PCT), which is computed by differentiable sampling from contact probabilities and used to generate motions. The contact state can be weakly supervised by encouraging the model to generate correct motions. Furthermore, we propose an attentive PD controller that adjusts target pose states using temporal information to obtain smooth and accurate pose control. Our approach is entirely neural-based and runs without offline optimization or simulation in physics engines. Experiments on large-scale 3D human motion benchmarks demonstrate the effectiveness of D &D, where we exhibit superior performance against both state-of-the-art kinematics-based and dynamics-based methods. Code is available at https://github.com/Jeff-sjtu/DnD .
T-Net: Deep Stacked Scale-Iteration Network for Image Dehazing

Haze reduces the visibility of image content and leads to failure in handling subsequent computer vision tasks. In this paper, we address the problem of single image dehazing by proposing a dehazing network named T-Net, which consists of a backbone network based on the U-Net architecture and a dual attention module. Multi-scale feature fusion can be achieved by using skip connections with a new fusion strategy. Furthermore, by repeatedly unfolding the plain T-Net, Stack T-Net is proposed to take advantage of the dependence of deep features across stages via a recursive strategy. To reduce network parameters, the intra-stage recursive computation of ResNet is adopted in our Stack T-Net. We take both the stage-wise result and the original hazy image as input to each T-Net and finally output the prediction of the clean image. Experimental results on both synthetic and real-world images demonstrate that our plain T-Net and the advanced Stack T-Net perform favorably against state-of-the-art dehazing algorithms and show that our Stack T-Net could further improve the dehazing effect, demonstrating the effectiveness of the recursive strategy.
Reduced-Space Multistream Classification Based on Multiobjective Evolutionary Optimization

The characteristics of data generators and real-world data streams are elaborated as follows:1) AGRAWAL generator [1]: It produces a stream with six numeric features, and three categorical ones.There are ten functions defined for generating binary class labels from the attributes.In the experiment, three functions are employed to simulate three kinds of concepts and form a data stream, denoted as AGRAWAL.A new concept completely replaces the old one over 10 instances at 5000th time step and 15000th time step to simulate sudden drift.Gradual drift occurs at the 10000th time step, where the concept changes within 1000 instances.
A Blind SAR Image Despeckling Method Based on Improved Weighted Nuclear Norm Minimization

Coherent imaging of synthetic aperture radar (SAR) systems generates multiplicative speckle noise, severely impairs SAR images’ interpretability. Although numerous despeckling methods have been proposed over the past three decades, SAR despeckling remains an unsolved problem due to its uniqueness and complexity. To address these issues, we propose a novel SAR image despeckling method based on the framework of weighted nuclear norm minimization (WNNM). First, a Gaussian function is used to approximate the distribution of good similar patches (GSP). Clustering is then used to select the optimal patches for the GSP matrix. WNNM can then estimate the underlying clean component of the GSP matrix. To increase the speed of the WNNM, a modified version of the singular value decomposition (SVD) is introduced. Additionally, we developed a noise variance estimation method that enables blind despeckling with the proposed method. Extensive experimental results demonstrate that the proposed method yields superior objective indices and subjective visual inspection.
Open-ended remote sensing visual question answering with transformers

Visual question answering (VQA) has been attracting attention in remote sensing very recently. However, the proposed solutions remain rather limited in the sense that the existing VQA datasets address closed-ended question-answer queries, which may not necessarily reflect real open-ended scenarios. In this paper, we propose a new dataset named VQA-TextRS that was built manually with human annotations and considers various forms of open-ended question-answer pairs. Moreover, we propose an encoder-decoder architecture via transformers on account of their self-attention property that allows relational learning of different positions of the same sequence without the need of typical recurrence operations. Thus, we employed vision and natural language processing (NLP) transformers respectively to draw visual and textual cues from the image and respective question. Afterwards, we applied a transformer decoder, which enables the cross-attention mechanism to fuse the earlier two modalities. The fusion vectors correlate with the process of answer generation to produce the final form of the output. We demonstrate that plausible results can be obtained in open-ended VQA. For instance, the proposed architecture scores an accuracy of 84.01% on questions related to the presence of objects in the query images.
Possible Utilization of Low Frequency Magnetic Fields in Short Range Multirotor UAV Detection System

The multirotor UAVs or so called n-copters have become very popular in the last decade and they have been utilized in many services. However, with the broaden options of buying multirotors, there is higher risk to the safety and security not only of the airports, but also of the urbanized areas. There are many UAV detection systems that have been developed in the world, the most sophisticated of them utilize multisensoric approach to fulfill the detection task. The paper presents chosen experimental results of the magnetic field measurements around a small quadcopter UAV that can serve as a proof of concept for possible utilization of low frequency magnetic fields in a short range multirotor UAV detection system. Although the fields created by the UAVs are small, they are clearly detectable by a simple coil sensor and in addition, they can be correlated with audio band signals to improve detection capabilities. The presented approach confirms the main advantage of the magnetic fields measurement - the possibility to detect a multirotor UAV operating in a completely autonomous mode without RF communication in urbanized areas where radars could not be used.
Fault Diagnosis in Industrial Control Networks Using Transferability-Measured Adversarial Adaptation Network

In recent years, the increasing number of industrial infrastructure security incidents around the world has drawn public attention to industrial control networks (ICNs) security issues. Fault diagnosis of industrial devices is an indispensable part of the security system in ICNs. The mainstream fault diagnosis models rely on long-term training and massive fault data, which results in the inability to update the model effectively and timely when the environment changes. Thus, some researchers focus on developing cross-domain industrial fault diagnosis methods. However, they usually presume that the samples of the target and source domains share the same fault mode sets, and existing prior knowledge concerning the label spaces of these two domains. These are difficult to satisfy in actual ICNs. To respond to these challenges, we develop a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">transferability-measured adversarial adaptation network</i> (TAAN) to identify unknown classes without prior knowledge. It embeds the hybrid transferability estimation into an adversarial domain adaptive network to weigh the contribution of each sample. In this way, TAAN can properly classify samples in a public label space by selectively aligning source and target samples with high transferability. The experimental results obtained using two diagnosis datasets prove that the developed TAAN can achieve satisfactory diagnostic accuracy by effectively bridging the distribution discrepancy under various working conditions.
Calibration-Aware Transpilation for Variational Quantum Optimization

Today's Noisy Intermediate-Scale Quantum (NISQ) computers support only limited sets of available quantum gates and restricted connectivity. Therefore, quantum algorithms must be transpiled in order to become executable on a given NISQ computer; transpilation is a complex and computationally heavy process. Moreover, NISQ computers are affected by noise that changes over time, and periodic calibration provides relevant error rates that should be considered during transpilation. Variational algorithms, which form one main class of computations on NISQ platforms, produce a number of similar yet not identical quantum ``ansatz'' circuits. In this work, we present a transpilation methodology optimized for variational algorithms under potentially changing error rates. We divide transpilation into three steps: (1) noise-unaware and computationally heavy pre-transpilation; (2) fast noise-aware matching; and (3) fast decomposition followed by heuristic optimization. For a complete run of a variational algorithm under constant error rates, only step (3) needs to be executed for each new ansatz circuit. Step (2) is required only if the error rates reported by calibration have changed significantly since the beginning of the computation. The most expensive Step (1) is executed only once for the whole run. This distribution is helpful for incremental, calibration-aware transpilation when the variational algorithm adapts its own execution to changing error rates. Experimental results on IBM's quantum computer show the low latency and robust results obtained by calibration-aware transpilation.
FMCW Radar-Based Hand Gesture Recognition Using Dual-Stream CNN-GRU Model

Data derived from mmWave radars, such as frequency modulated continuous wave (FMCW) radars, contains distinctive movement-based features that characterize each gesture uniquely and facilitate contactless human hand gesture recognition. This paper aims to use an advanced 77-GHz multiple-input-multiple-output (MIMO) FMCW radar with a deep-learning model to automate the extraction of these unique features. We forward this radar’s pre-processed range-Doppler and range-angle images (RDIs and RAIs) into a dual-stream artificial neural network to classify human hand gestures. The proposed multiple-input, single-output architecture comprises 2D convolutional neural network-gated recurrent units (2D CNNGRU). According to the conducted experiments, the average accuracy of the proposed classification model with 8-fold cross-validation achieves 92.50%.
Crypto-Dropout: To Create Unique User-Generated Content Using Crypto Information in Metaverse

In a blockchain-driven metaverse, user-generated content (UGC) is the core power for building the metaverse, so an easy-to-use UGC editor is imperative. Specifically, using artificial intelligence (AI) to simplify the UGC creation procedure is promising, e.g., generating images from sketches using generative adversarial networks (GANs). However, the simplicity of these UGC creation methods would lead to weak distinctions between the generated UGC, since the users' created drafts may be very similar. In this paper, we propose Crypto-dropout, a specially designed dropout used in the generative neural networks, which could cause pseudo-random disturbance based on the hash value of user information to generate unique results. With a pilot study, the experimental results demonstrate that the participants have different preferences for the generated images when setting Crypto-dropout in the different layers. Accordingly, we implement a practical profile pictures (PFPs) creation prototype. The proposed Crypto-dropout can provide a novel and general insight for creating unique UGC using generative neural networks.
Defuse the Training of Risky Tasks: Collaborative Training in XR

Extensive training is crucial but challenging in certain areas such as explosive ordnance disposal. Past conflicts have shown that not only military personal but also civilians have to learn how to disarm unexploded ordnance. The preparation for dangerous situations is difficult and limited in the real world. Extended reality (XR) offers new possibilities to enhance the training of explosive ordnance disposal experts due to its immersive capabilities. This paper presents a comparative study (n = 75) of three distinct training methods: 1) Real-world (Real)-Training with a tangible replica object, 2) Virtual Reality (VR)-Training with a non-see-through Head-Mounted-Display (HMD), and 3) Mixed Reality (MR)-Training in a Cave Automatic Virtual Environment (CAVE). All training methods are collaborative, i.e., an instructor teaches the training content to a participant in the real or virtual world. We evaluate the suitability of these approaches in terms of usability, cognitive workload, training motivation, and training success. Our results indicate that the virtual methods, VR-Training and MR-Training, provide significantly superior results in the evaluated aspects compared to the real-world training. These results can also be applied to other collaborative training methods, as the training concept of this use case was non-specific. Therefore, these virtual technologies can increase the safety of explosive ordnance disposal personnel, and we recommend establishing this in future training.
Reducing Background Induced Domain Shift for Adaptive Person Re-Identification

Cross-domain person re-identification (Re-ID) is a challenging and important task in monitoring safety and procedure compliance of industrial work places. In this article, a novel method is proposed to reduce background induced domain shift for adaptive person Re-ID. Specifically, a foreground-background joint clustering module is proposed to extract discriminative foreground and background features and an attention-based feature disentanglement module is designed to reduce the interference of background with the extraction of discriminative foreground features. Experimental results on three widely used person Re-ID benchmarking datasets (Market-1501, DukeMTMC-reID, and MSMT17) have demonstrated that the proposed method achieves promising performance compared with the state-of-the-art methods.
VirtualClassroom: A Lecturer-Centered Consumer-Grade Immersive Teaching System in Cyber–Physical–Social Space

Lecturers, as the guidance of the classroom, play a significant role in the teaching process. However, the lecturers’ sense of space immersion has been ignored in current virtual teaching systems. In this article, we explore the cyber–physical–social intelligence for Edu-Metaverse in cyber–physical–social space and specially design a lecturer-centered immersive teaching system, taking the social and lecturers’ factors into consideration. We call this system VirtualClassroom (V-Classroom). Specifically, we first introduce the cyber–physical–social system (CPSS) paradigm of V-Classroom so that the workflow is standardized and significantly simplified, and the systems can be constructed with off-the-shelf hardware. The key component of V-Classroom is a cyber-world representation of a physical-world classroom instrumented with sparse consumer-grade RGBD cameras for capturing the 3-D geometry and texture of the classrooms. We provide each V-Classroom lecturer with a physical device for sending 6DoF view-change messages and showing view-dependent content of the remote classroom. Following the above paradigm, we develop the V-Classroom algorithms, including V-Classroom depth algorithm (V-DA) and V-Classroom view algorithm (V-VA), to achieve the real-time rendering of remote classrooms. V-DA is dedicated to recovering accurate depth information of the classrooms while V-VA is devoted to real-time novel view synthesis. Finally, we illustrate our implemented CPSS-driven V-Classroom prototype, based on real-world classroom scenarios we collected, and discuss the main challenges and future direction.
Spatial-Temporal Parallel Transformer for Arm-Hand Dynamic Estimation

We propose an approach to estimate arm and hand dynamics from monocular video by utilizing the relationship between arm and hand. Although monocular full human motion capture technologies have made great progress in recent years, recovering accurate and plausible arm twists and hand gestures from in-the-wild videos still remains a challenge. To solve this problem, our solution is proposed based on the fact that arm poses and hand gestures are highly correlated in most real situations. To fully exploit arm-hand correlation as well as inter-frame information, we carefully design a Spatial-Temporal Parallel Arm-Hand Motion Transformer (PAHMT) to predict the arm and hand dynamics simultaneously. We also introduce new losses to encourage the estimations to be smooth and accurate. Besides, we collect a motion capture dataset including 200K frames of hand gestures and use this data to train our model. By integrating a 2D hand pose estimation model and a 3D human pose estimation model, the proposed method can produce plausible arm and hand dynamics from monocular video. Extensive evaluations demonstrate that the proposed method has advantages over previous state-of-the-art approaches and shows robustness under various challenging scenarios.
Structure-Preserving Texture Smoothing via Scale-Aware Bilateral Total Variation

The purpose of texture smoothing is to preserve the prominent structure in the image while smoothing the salient texture. However, the existing methods are difficult to achieve a satisfactory balance between filtering out salient textures and preserving weak edge structures and small structures. To this end, we propose a structure-preserving texture smoothing method via scale-aware bilateral total variation. First, the joint bilateral filter is introduced to construct the window bilateral variation, and combined with the window total variation, a regularizer called the bilateral total variation is formed, which accurately quantifies the characteristics of texture and structure, to finely smooth salient textures while preserving weak edge structures and small structures. Subsequently, we proposed a scale-aware scheme to make the proposed regularizer more powerful in preserving small structures and adopted an optimization scheme to convert the original non-convex optimization problem into a least squares regression problem. The effectiveness of the proposed regularizer is verified in the dataset. The experimental results demonstrate the superiority of the proposed method in texture smoothing and other applications compared to other state-of-the-art approaches.
Systematic Literature Review on The Usage of IoT in The Metaverse to Support The Education System

The education process can be done in various methods. With the current developing technology, education methods have become more efficient than traditional learning. Remote learning is one of the applications in which modern technology is used to support the education system. Although despite all the advancements, the current remote learning system still has its limitations such as the lack of interactivity and immersive. Metaverse is one of the newer technology applications in education along with IoT. Metaverse bridges the real and virtual worlds and can provide an immersive and stimulating learning experience. This paper provides a review of the integration of IoT in the metaverse for a better learning experience. This paper uses the systematic literature review method to find results regarding the research questions and draw a conclusion. From the result of this paper, it can be concluded that IoT and Metaverse are individually beneficial for the education system, and integrating IoT into the metaverse can help with data mapping to give a more accurate result of digital twin and build an immersive ecosystem for a more effective learning environment.
Speculative Multipliers on DeFi: Quantifying On-Chain Leverage Risks

Blockchains and DeFi have consistently shown to attract financial speculators. One avenue to increase the potential upside (and risks) of financial speculation is leverage trading, in which a trader borrows assets to participate in the financial market. While well-known over-collateralized loans, such as MakerDAO, only enable leverage multipliers of $$1.67\times $$ , new under-collateralized lending platforms, such as Alpha Homora (AH), unlock leverage multipliers of up to $$8\times $$ and attracted over 1.2B $$ \texttt {USD}$$ of locked value at the time of writing. In this paper, we are the first to formalize a model for under-collateralized DeFi lending platforms. We analytically exposit and empirically evaluate the three main risks of a leverage-engaging borrower: (i) impermanent loss (IL) inherent to Automated Market Makers (AMMs), (ii) arbitrage loss in AMMs, and (iii) collateral liquidation. Based on our analytical and empirical results of AH over a timeframe of 9 months, we find that a borrower may mitigate the IL through a high leverage multiplier (e.g., more than $$4\times $$ ) and a margin trading before supplying borrowed assets into AMMs. We interestingly find that the arbitrage and liquidation losses are proportional to the leverage multiplier. In addition, we find that 72.35% of the leverage taking borrowers suffer from a negative APY, when ignoring the governance token incentivization in AH. Finally, when assuming a maximum $$\pm 10$$ % move among two stablecoins, we pave the way for more extreme on-chain leverage multipliers of up to $$91.9\times $$ by providing appropriate system settings.