Reviewerly/exHarmony huggingface dataset:
Desynchronization Attacks Resilient Watermarking Method Based on Frequency Singular Value Coefficient Modification

Desynchronization is a very challenging type of attack in audio watermarking. The traditional singular value decomposition (SVD) based audio watermarking methods embed the watermark information by modifying the singular value of individual segment, which have little resistance against desynchronization attacks. In this paper, we propose a novel frequency singular value coefficient (FSVC) feature, which reflects the ratio between the singular values of two consecutive segments and is insensitive to desynchronization attacks, to carry the watermark bits. To our best knowledge, it is the first time that the ratio between singular values is employed for audio watermarking. In the proposed method, the discrete cosine transform (DCT) is performed on two consecutive segments of the host audio signal and SVD is applied to the DCT coefficients of the mid frequency band of each segment to extract the FSVC. Then the watermark bits are embedded by adjusting the values of the FSVC. The watermark embedding procedure is optimized to minimize the perceptual quality degradation and an error buffer is created to enhance the robustness. As a result, the proposed method can achieve a much higher embedding capacity than the existing methods tackling desynchronization attacks. The impact of desynchronization and common signal processing attacks on the proposed watermarking method is mathematically modeled, and the effectiveness of the proposed method against these attacks is theoretically and experimentally validated.
PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency

Different from general photo retouching tasks, portrait photo retouching (PPR), which aims to enhance the visual quality of a collection of flat-looking portrait photos, has its special and practical requirements such as human-region priority (HRP) and group-level consistency (GLC). HRP requires that more attention should be paid to human regions, while GLC requires that a group of portrait photos should be retouched to a consistent tone. Models trained on existing general photo retouching datasets, however, can hardly meet these requirements of PPR. To facilitate the research on this high-frequency task, we construct a largescale PPR dataset, namely PPR10K, which is the first of its kind to our best knowledge. PPR10K contains 1, 681 groups and 11, 161 high-quality raw portrait photos in total. High-resolution segmentation masks of human regions are provided. Each raw photo is retouched by three experts, while they elaborately adjust each group of photos to have consistent tones. We define a set of objective measures to evaluate the performance of PPR and propose strategies to learn PPR models with good HRP and GLC performance. The constructed PPR10K dataset provides a good bench-mark for studying automatic PPR methods, and experiments demonstrate that the proposed learning strategies are effective to improve the retouching performance. Datasets and codes are available: https://github.com/csjliang/PPR10K.
Efficient Federated-Learning Model Debugging

Federated learning (FL) enables large amounts of participants to construct a global learning model, while storing training data privately at each client device. A fundamental issue in this framework is the susceptibility to the erroneous training data. This problem is especially challenging due to the invisibility of clients' local training data and training process, as well as the resource constraints of a large number of mobile and edge devices. In this paper, we try to tackle this challenging issue by introducing the first FL debugging framework, FLDebugger, for mitigating test error caused by erroneous training data. The pro-posed solution traces the global model's bugs (test errors), jointly through the training log and the underlying learning algorithm, back to first identify the clients and subsequently their training samples that are most responsible for the errors. In addition, we devise an influence-based participant selection strategy to fix bugs as well as to accelerate the convergence of model retraining. The performance of the identification algorithm is evaluated via extensive experiments on a real AIoT system (50 clients, including 20 edge computers, 20 laptops and 10 desktops) and in larger-scale simulated environments. The evaluation results attest to that our framework achieves accurate and efficient identification of negatively influential clients and samples, and significantly improves the model performance by fixing bugs.
PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction

Human motion prediction, which aims to predict future human poses given past poses, has recently seen increased interest. Many recent approaches are based on Recurrent Neural Networks (RNN) which model human poses with exponential maps. These approaches neglect the pose velocity as well as temporal relation of different poses, and tend to converge to the mean pose or fail to generate natural-looking poses. We therefore propose a novel Position-Velocity Recurrent Encoder-Decoder (PVRED) for human motion prediction, which makes full use of pose velocities and temporal positional information. A temporal position embedding method is presented and a Position-Velocity RNN (PVRNN) is proposed. We also emphasize the benefits of quaternion parameterization of poses and design a novel trainable Quaternion Transformation (QT) layer, which is combined with a robust loss function during training. We provide quantitative results for both short-term prediction in the future 0.5 seconds and long-term prediction in the future 0.5 to 1 seconds. Experiments on several benchmarks show that our approach considerably outperforms the state-of-the-art methods. In addition, qualitative visualizations in the future 4 seconds show that our approach could predict future human-like and meaningful poses in very long time horizons. Code is publicly available on GitHub: https://github.com/hongsong-wang/PVRNN.
Towards A Process Model for Co-Creating AI Experiences

Thinking of technology as a design material is appealing. It encourages designers to explore the material's properties to understand its capabilities and limitations—a prerequisite to generative design thinking. However, as a material, AI resists this approach because its properties only emerge as part of the user experience design. Therefore, designers and AI engineers must collaborate in new ways to create both the material and its application experience. We investigate the co-creation process through a design study with 10 pairs of designers and engineers. We find that design 'probes' with user data are a useful tool in defining AI materials. Through data probes, designers construct designerly representations of the envisioned AI experience (AIX) to identify desirable AI characteristics. Data probes facilitate divergent design thinking, material testing, and design validation. Based on our findings, we propose a process model for co-creating AIX and offer design considerations for incorporating data probes in AIX design tools.
Masked Face Recognition Using Principal component analysis and Deep learning

After COVID-19 pandemic most people all around the world are wearing face masks in public which makes traditional face recognition methods insufficient to recognize a face with mask. According to NIST2020 [1] report, face recognition accuracy of existing face recognition methods has a 20-50% of error rate due to the face covered by face mask, therefore, video surveillance systems and other face authentication systems are now not working efficiently. Recently many researchers have tried to solve this problem using mainly deep learning methods (CNN, FaceNet). Some researchers have used features from occluded (area covered by face mask) part and non-occluded part of the face while some researchers discard the occluded part and only used the non-occluded part of the face to recognize. For both of these two cases, the facial features of the occluded part of the face cannot be analyzed or cannot be used to recognize the face, as a result, these methods have high error rate. This paper proposed a reconstructive method to get partially reconstructed features of the occluded part of the face, then existing deep learning method has used to recognize the face. First the occluded part of a face is discarded and Principal component analysis (PCA) is applied to the remaining part of the face. Next, the principal components or the most significant Eigenvector and its corresponding weights are calculated to reconstruct the occluded part. Proposed method cannot reconstruct the occluded part completely but it can provide enough features of the occluded part which improves the recognition accuracy up to 15%.
Learning to Associate Every Segment for Video Panoptic Segmentation

Temporal correspondence - linking pixels or objects across frames - is a fundamental supervisory signal for the video models. For the panoptic understanding of dynamic scenes, we further extend this concept to every segment. Specifically, we aim to learn coarse segment-level matching and fine pixel-level matching together. We implement this idea by designing two novel learning objectives. To validate our proposals, we adopt a deep siamese model and train the model to learn the temporal correspondence on two different levels (i.e., segment and pixel) along with the target task. At inference time, the model processes each frame independently without any extra computation and post-processing. We show that our per-frame inference model can achieve new state-of-the-art results on Cityscapes-VPS and VIPER datasets. Moreover, due to its high efficiency, the model runs in a fraction of time (3×) compared to the previous state-of-the-art approach.
Infrared Thermal Imaging Super-Resolution via Multiscale Spatio-Temporal Feature Fusion Network

Infrared thermal imaging is widely used in military and civil fields like security and protecting monitoring, military reconnaissance, road traffic monitoring. However, most current infrared sensors have limited resolution for their technological limitation. To solve it, we propose a novel end-to-end infrared thermal imaging super-resolution via multiscale spatio-temporal feature fusion network, that can fuse the temporal aligned features of the neighboring frames and the spatial feature of the reference frame. In our network, a pyramid multiscale fusion module is designed as major kernel of the whole structure, which could release the spatio-temporal misalignment between the neighboring frames and the reference frame. Specifically, we introduce the channel attention mechanism to further improve the infrared thermal imaging video super-resolution performance. Qualitative and quantitative evaluation results prove the effectiveness of our framework on both Vid4 dataset and raw infrared dataset collected by ourselves. We provide the infrared thermal imaging super-resolution results at https://github.com/qq123-jpg/Infrared-Thermal-Imaging-Super-resolution-via-Multiscale-Spatio-temporal-Feature-Fusion-Network.
Partial Feature Selection and Alignment for Multi-Source Domain Adaptation

Multi-Source Domain Adaptation (MSDA), which dedicates to transfer the knowledge learned from multiple source domains to an unlabeled target domain, has drawn increasing attention in the research community. By assuming that the source and target domains share consistent key feature representations and identical label space, existing studies on MSDA typically utilize the entire union set of features from both the source and target domains to obtain the feature map and align the map for each category and domain. However, the default setting of MSDA may neglect the issue of "partialness", i.e., 1) a part of the features contained in the union set of multiple source domains may not present in the target domain; 2) the label space of the target domain may not completely overlap with the multiple source domains. In this paper, we unify the above two cases to a more generalized MSDA task as Multi-Source Partial Domain Adaptation (MSPDA). We propose a novel model termed Partial Feature Selection and Alignment (PFSA) to jointly cope with both MSDA and MSPDA tasks. Specifically, we firstly employ a feature selection vector based on the correlation among the features of multiple sources and target domains. We then design three effective feature alignment losses to jointly align the selected features by preserving the domain information of the data sample clusters in the same category and the discrimination between different classes. Extensive experiments on various benchmark datasets for both MSDA and MSPDA tasks demonstrate that our proposed PFSA approach remarkably outperforms the state-of-the-art MSDA and unimodal PDA methods.
Dense Point Prediction: A Simple Baseline for Crowd Counting and Localization

In this paper, we propose a simple yet effective crowd counting and localization network named SCALNet. Unlike most existing works that separate the counting and localization tasks, we consider those tasks as a pixel-wise dense prediction problem and integrate them into an end-to-end framework. Specifically, for crowd counting, we adopt a counting head supervised by the Mean Square Error (MSE) loss. For crowd localization, the key insight is to recognize the keypoint of people, i.e., the center point of heads. We propose a localization head to distinguish dense crowds trained by two loss functions, i.e., Negative-Suppressed Focal (NSF) loss and False-Positive (FP) loss, which balances the positive/negative examples and handles the false-positive predictions. Experiments on the recent and large-scale benchmark, NWPU-Crowd, show that our approach outperforms the state-of-the-art methods by more than 5% and 10% improvement in crowd localization and counting tasks, respectively. The code is publicly available at https://github.com/WangyiNTU/SCALNet.
Robust Vehicle Detection in High-Resolution Aerial Images With Imbalanced Data

Vehicle detection in images from unmanned aerial vehicles (UAVs) plays an important role in traffic surveillance and urban planning due to the popularity of UAVs. However, the class imbalance problem is an important factor that restricts the performance of vehicle detectors. There are two types of class imbalance in UAV images, i.e., foreground-background imbalance and foreground–foreground imbalance. For anchor-based single stage detector, as many ground truths cannot be assigned to corresponding anchors because of low intersection over union, it makes the foreground-background imbalance problem more severe. Therefore, we propose a novel bag-based single-stage detector, which treats each position on the feature map as a bag. A simple and adaptive definition of bags is proposed along with the positive sample definition method, which is utilized to ensure more ground truths can be assigned to proper bags. In addition, we utilize online hard example mining method to control the proportion of positive and negative samples during the training process. To address the foreground–foreground imbalance, we propose a novel data augmentation algorithm, which allows us to create appropriate visual context for under-represented class. Extensive experiments demonstrate the superiority of the proposed algorithm, compared with other state-of-the-art solutions. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement—Recently, unmanned aerial vehicles (UAVs) are widely used in intelligent transportation due to their low price and high flexibility, which makes vehicle detection in UAV images important for automatically gathering of traffic information. However, the class imbalance problem, which is common in object detection where some classes have far fewer frequencies in the dataset, has an adverse effect on the performance of vehicle detectors. The data augmentation method and deep learning based vehicle detector proposed in this article are able to reduce the negative impact and improve detection performance by at least 1.27% in mean average precision index. In addition, compared with algorithms with similar detection performance, our method is at least 15 ms faster. The proposed method can benefit users in a wide variety of applications including UAV transportation, traffic surveillance, and urban planning.
Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay

This paper proposes two novel knowledge transfer techniques for class-incremental learning (CIL). First, we propose data-free generative replay (DF-GR) to mitigate catastrophic forgetting in CIL by using synthetic samples from a generative model. In the conventional generative replay, the generative model is pre-trained for old data and shared in extra memory for later incremental learning. In our proposed DF-GR, we train a generative model from scratch without using any training data, based on the pre-trained classification model from the past, so we curtail the cost of sharing pre-trained generative models. Second, we introduce dual-teacher information distillation (DT-ID) for knowledge distillation from two teachers to one student. In CIL, we use DT-ID to learn new classes incrementally based on the pre-trained model for old classes and another model (pre-)trained on the new data for new classes. We implemented the proposed schemes on top of one of the state-of-the-art CIL methods and showed the performance improvement on CIFAR-100 and ImageNet datasets.
Cryo-CMOS for Quantum System On-Chip Integration: Quantum Computing as the Development Driver

Quantum computing (QC) is a new paradigm that exploits fundamental principles of quantum mechanics, such as superposition and entanglement, to tackle problems in mathematics, chemistry, and material science that are well beyond the reach of supercomputers [1]. Its power is derived from a quantum bit (qubit) that can simultaneously exist in a superposition of both zero and one states and become entangled with other qubits. It has been shown that quantum computers can speed up algorithms and, potentially, model any physical process [2]. QC is now the main driver behind the phenomenal development of cryogenic CMOS, or cryo-CMOS [3]. The starting point was the sudden emergence of rudimentary quantum computers from obscure physics labs [4]. In the past, there was simply no sufficient motivation to go through the pain of designing IC circuits at deep cryogenic temperatures, where the silicon substrate is known to freeze out electrically and where no Spice models exist. Hence, we cannot find any remarkable literature related to cryo-CMOS from back then. Now that cryo-CMOS work has seriously begun, the designed circuits are being deployed for other applications, such as astronomy and physics experiments [5]-[12].
Improved high-degree cubature Kalman filter based on resampling-free sigma-point update framework and its application for inertial navigation system-based integrated navigation

Sigma-point based Kalman filters have been widely applied to nonlinear filtering problems, where cubature Kalman filter (CKF) is considered to be an ideal solution to tightly coupled GNSS/INS. Different sigma-point update frameworks (SUFs) for CKF have been reported in our previous work, where instead of being abandoned after the moment matching, the instantiated points are used to approximate the likelihood function and update the posterior sigma points directly, which leads to a resampling-free SUF. By propagating more cubature points through the filtering model, the High-degree CKF (HCKF) achieves more accurate moments approximation result than CKF. In this paper, the performance of different SUFs for HCKF is analyzed by simulation on tightly coupled GNSS/INS, and observations missing during different maneuvers are simulated to evaluate the performance of resampling-free HCKF. The simulation result gives an insight into the uncertainty propagation in the two-stage update of Gaussian filter for GNSS/INS application.
Avoidance of Cybersecurity Threats with the Deployment of a Web-Based Blockchain-Enabled Cybersecurity Awareness System

Modern information technology (IT) is well developed, and almost everyone uses the features of IT and services within the Internet. However, people are being affected due to cybersecurity threats. People can adhere to the recommended cybersecurity guidelines, rules, adopted standards, and cybercrime preventive measures to largely mitigate these threats. The ignorance of or lack of cybersecurity knowledge also causes a critical problem regarding confidentiality and privacy. It is not possible to fully avoid cybercrimes that often lead to sufficient business losses and spread forbidden themes (disgust, extremism, child porn, etc.). Therefore, to reduce the risk of cybercrimes, a web-based Blockchain-enabled cybersecurity awareness program (WBCA) process is introduced in this paper. The proposed WBCA trains users to improve their security skills. The proposed program helps with understanding the common behaviors of cybercriminals and improves user knowledge of cybersecurity hygiene, best cybersecurity practices, modern cybersecurity vulnerabilities, and trends. Furthermore, the proposed WBCA uses Blockchain technology to protect the program from potential threats. The proposed program is validated and tested using real-world cybersecurity topics with real users and cybersecurity experts. We anticipate that the proposed program can be extended to other domains, such as national or corporate courses, to increase the cybersecurity awareness level of users. A CentOS-based virtual private server is deployed for testing the proposed WBCA to determine its effectiveness. Finally, WBCA is also compared with other state-of-the-art web-based programs designed for cybersecurity awareness.
Futures Studies, Mobilities, and the Postdigital Condition: Contention or Complement

This paper explores relationships between recent developments in the fields of mobilities, futures, and postdigital studies. The article covers six main themes: questions and their histories; definitions; research methods and ethics; the nature and ownership of knowing and learning; understandings of time, space, identity, community, and relationships; and political processes and political legitimacy. The article was written in three steps. In the first step, the leading author (John Traxler) has identified the relevant themes. In the second step, proponents of each position have freely responded to the themes (futures studies, Stuart Connor; postdigital theory, Sarah Hayes and Petar Jandrić; mobilities, John Traxler). In the third step, the responses have been collectively (re)mixed and edited, identifying complementary and conflicting concepts and ideas. The article was initiated a month before the outbreak of the Covid-19 pandemic, and it was completed over one and a half years later. Thusly, responses and analyses have included the pandemic experience without explicitly focusing to the Covid-19 pandemic. The paper concludes with drawing together contributions, seeking underlying commonalities and differences, and looking for trends, convergence, and change. Epistemically, the three positions discussed in this paper are far from commensurable. Yet they are compatible and complementary, in a postdigital dialogue, in a sense that they all need each others' inputs on the road to a better understanding of our current condition, and the road to a better future.