Reviewerly/exHarmony huggingface dataset:
STATE: Learning structure and texture representations for novel view synthesis

Abstract Novel viewpoint image synthesis is very challenging, especially from sparse views, due to large changes in viewpoint and occlusion. Existing image-based methods fail to generate reasonable results for invisible regions, while geometry-based methods have difficulties in synthesizing detailed textures. In this paper, we propose STATE, an end-to-end deep neural network, for sparse view synthesis by learning structure and texture representations. Structure is encoded as a hybrid feature field to predict reasonable structures for invisible regions while maintaining original structures for visible regions, and texture is encoded as a deformed feature map to preserve detailed textures. We propose a hierarchical fusion scheme with intra-branch and inter-branch aggregation, in which spatio-view attention allows multi-view fusion at the feature level to adaptively select important information by regressing pixel-wise or voxel-wise confidence maps. By decoding the aggregated features, STATE is able to generate realistic images with reasonable structures and detailed textures. Experimental results demonstrate that our method achieves qualitatively and quantitatively better results than state-of-the-art methods. Our method also enables texture and structure editing applications benefiting from implicit disentanglement of structure and texture. Our code is available at http://cic.tju.edu.cn/faculty/likun/projects/STATE .
Enhancing Voice Cloning Quality through Data Selection and Alignment-Based Metrics

Voice cloning, an emerging field in the speech-processing area, aims to generate synthetic utterances that closely resemble the voices of specific individuals. In this study, we investigated the impact of various techniques on improving the quality of voice cloning, specifically focusing on a low-quality dataset. To contrast our findings, we also used two high-quality corpora for comparative analysis. We conducted exhaustive evaluations of the quality of the gathered corpora in order to select the most-suitable data for the training of a voice-cloning system. Following these measurements, we conducted a series of ablations by removing audio files with a lower signal-to-noise ratio and higher variability in utterance speed from the corpora in order to decrease their heterogeneity. Furthermore, we introduced a novel algorithm that calculates the fraction of aligned input characters by exploiting the attention matrix of the Tacotron 2 text-to-speech system. This algorithm provides a valuable metric for evaluating the alignment quality during the voice-cloning process. We present the results of our experiments, demonstrating that the performed ablations significantly increased the quality of synthesised audio for the challenging low-quality corpus. Notably, our findings indicated that models trained on a 3 h corpus from a pre-trained model exhibit comparable audio quality to models trained from scratch using significantly larger amounts of data.
Performance analysis of H2BR: HTTP/2-based segment upgrading to improve the QoE in HAS

Abstract HTTP Adaptive Streaming (HAS) plays a key role in over-the-top video streaming with the ability to reduce the video stall duration by adapting the quality of transmitted video segments to the network conditions. However, HAS still suffers from two problems. First, it incurs variations in video quality because of throughput fluctuation. Adaptive bitrate (ABR) algorithms at the HAS client usually select a low-quality segment when the throughput drops to avoid stall events, which impairs the Quality of Experience (QoE) of the end-users. Second, many ABR algorithms choose the lowest-quality segments at the beginning of a video streaming session to ramp up the playout buffer early on. Although this strategy decreases the startup time, clients can be annoyed as they have to watch a low-quality video initially. To address these issues, we introduced the H2BR technique ( H TTP/ 2 - B ased R etransmission) (Nguyen et al. 33) that utilizes certain features of HTTP/2 (including server push , multiplexing , stream priority , and stream termination ) for late transmissions of higher-quality versions of video segments already in the client buffer, in order to improve video quality. Although H2BR was shown to enhance the QoE, limited streaming scenarios were considered resulting in a lack of general conclusions on H2BR’s performance. Thus, this article provides a profound evaluation to answer three open questions: (i) how H2BR’s performance is impacted by parameters at the server side (i.e., various encoding specifications), at the network side (i.e., packet loss rate), and at the client side (i.e., buffer size) on the performance of H2BR; (ii) how H2BR outperforms other state-of-the-art approaches in different configurations of the parameters above; (iii) how to effectively utilize H2BR on top of ABR algorithms in various streaming scenarios. The experimental results show that H2BR’s performance increases with the buffer size and decreases with increasing packet loss rates and/or video segment duration. The number of quality levels can negatively or positively impact on H2BR’s performance, depending on the ABR algorithm deployed. In general, H2BR is able to enhance the video quality by up to 17% and 14% in scalable video streaming and in non-scalable video streaming, respectively. Compared with an existing retransmission technique (i.e., SQUAD Wang et al., ACM Trans Multimed Comput Commun Applic (TOMM) 13(3s): 45, 49), H2BR shows better results with more than 10% in QoE and 9% in the average video quality.
First-order primal–dual algorithm for image restoration corrupted by mixed Poisson–Gaussian noise

The total variation infimal convolution (TV-IC) model combining Kullback–Leibler and ℓ2-norm data fidelity term works well for the inverse problem of mixed Poisson–Gaussian noise. Most existing algorithms for solving the TV-IC model rely on the Newton method to solve a nonlinear optimization subproblem, which inevitably increases the computation burden. In this study, we apply the first-order primal–dual Chambolle-Pock algorithm to solve the TV-IC model. In particular, we present an effective algorithm to solve the subproblem of the joint proximal operator with the Kullback–Leibler divergence and ℓ2-norm, which is based on the bilinear constraint alternating direction multiplier method. Numerical experiment results demonstrate that the proposed algorithm outperforms the state-of-the-art methods for mixed Poisson–Gaussian denoising and deblurring problems.
Scale-progressive Multi-patch Network for image dehazing

Image dehazing is a classical vision task, which aims to recover a clean image from a hazy one. Previous dehazing methods usually follow a coarse-to-fine architecture to mine clean features by introducing generic CNNs components. However, this manner usually results in undesirable model complexity and computational burden. In this work, we present a Scale-progressive Multi-patch Network (SPM-Net) to handle common problems in previous dehazing networks. Specifically, our approach utilizes a scale-progressive multi-patch mechanism to efficiently model uneven hazy distribution on local patches and progressively explore clean cues in multiple scales in a fine-to-coarse way. Besides of above, we found the feature misalignment problem in the patch-based methods, and a practical solution is proposed to handle this previously neglected problem. A comprehensive evaluation of both synthetic datasets and real-world datasets demonstrates that the proposed dehazing method surpasses the previous state-of-the-art approaches with a margin both quantitatively and qualitatively. Our proposed SPM-Net achieved a PSNR of 29.47 dB on the Haze4k dataset, significantly surpassing the previous state-of-the-art method DMT-Net (29.47 dB vs. 28.53 dB) while having much fewer parameters than DMT-Net (16.1 M vs. 54.9 M) and faster inferencing efficiency (0.072 s vs. 0.192 s).
Prolonging the Qubits Entanglement via Local Surface Plasmon Modes

A key to achieving quantum computing is the qubit-entanglement. Nevertheless, real-life qubits cannot be isolated from the environment completely. As a result, the entanglement between qubits decoheres via their interaction with the environment. Therefore, it is essential to understand how the qubit-entanglement changes over time. In this work, we study the entanglement between qubits when placed in a bulk of dielectric material embedded with a nanosized metal particle. Most importantly, we focus on the effects of the unavoidable localized surface plasmons induced at the surface of the metal particle. We discuss how the distance between the metal particles affects the entanglement of the qubits and how various entangled states are affected by the localized surface plasmons.
Multimodal query-guided object localization

Recent studies have demonstrated the effectiveness of using hand-drawn sketches of objects as queries for one-shot object localization. However, hand-drawn crude sketches alone can be ambiguous for object localization, which could result in misidentification, e.g., a sketch of a laptop could be confused for a sofa. To overcome this, we propose a novel multimodal approach to object localization that combines sketch queries with linguistic category definitions, allowing for a better representation of visual and semantic cues. Our approach employs a cross-modal attention scheme that guides the region proposal network to obtain relevant proposals. Further, we propose an orthogonal projection-based proposal scoring technique that effectively ranks proposals with respect to the query. We evaluated our method using hand-drawn sketches from the ‘Quick, Draw!’ dataset and glosses from ‘WordNet’ as queries on the widely-used MS-COCO dataset, and achieve superior performance compared to related baselines in both open- and closed-set settings.
An Image Stitching Scheme Based on Cameras Coordinate Transformation and Template Matching

Abstract In the detection of a large-size longitudinal beam workpiece, it is impossible to obtain the complete picture of the workpiece only by relying on a single camera, so it is necessary to realize the position movement of the camera with the help of a motor to obtain image data one by one, in which image stitching is the key to realize this technology. Based on this, this paper proposes an image stitching scheme based on the camera’s coordinate transformation and template matching. The scheme first uses a motor to drive three cameras to the image acquisition and then uses the coordinate transformation between cameras to obtain the stitched image of the area. After cameras are moved to the next image acquisition area, the image stitching algorithm based on template matching is used to complete the image stitching of the two areas. The above steps are repeated until the cameras complete the image acquisition task of all areas and the image stitching of the final workpiece is completed. Through the experimental verification, the proposed template matching scheme for the approximate overlapping areas between images can effectively improve the efficiency of image stitching, and the specific improvement effect in this experiment can reach 46.2%, and the overall image stitching scheme meets the requirements of detection efficiency.
Perspective Chapter: The Efforts to Increase Students’ Understanding of Prayer through the Use of Audiovisual Media during the COVID-19 Pandemic in Elementary Schools

Audio-visual media is an effective tool for teaching prayer to grade I children due to their preference for visual stimuli and ability to imitate movements seen in videos. This approach can replace direct classroom instruction and promote better learning outcomes. The research employed class action research at SD Ar-Rahman Motik, South Jakarta, specifically in class IB. The aim was to enhance students’ understanding of prayer. The study began with pre-cycle activities to assess students’ abilities, followed by teaching prayer using audio-visual media during the cycle. The results demonstrated improvement after the implementation of audio-visual media. Prior to its use, only 11% of students demonstrated a good understanding of prayer readings, while 89% had sufficient or lacking ability. In terms of prayer movements, 5% exhibited good proficiency, and 95% had sufficient or lacking ability, falling short of the Minimum Completeness Criteria. However, after the intervention, 94% achieved good proficiency (39% good and 55% very good) in prayer readings, and 100% exhibited good proficiency (39% good and 61% very good) in prayer movements. Only 6% (1 child) showed a deficiency in prayer reading. Overall, 80% of students met the criteria, eliminating the need for Cycle II.
Security enhancement of LSB-based audio steganography method

The rapid development of digital technology has brought about the challenge of ensuring information security. Cryptography and steganography are among the various techniques available to address this challenge. These techniques come in different forms and provide reliable means of securing information and communication. Despite the existence of numerous options, new variations are still emerging. This paper concentrates on Least Significant Bit (LSB) coding, which is one of the most widely recognized and frequently utilized steganography techniques. The advantages of LSB-based coding methods are high capacity and low complexity. However, predictability is an issue. Audio files’ sizes and data redundancy make audio data perfect for steganography, as it is possible to embed large amounts of secret information and easily transmit the signal via various communication channels. However, the use of LSB coding in audio steganography has a drawback due to the extreme sensitivity of the human auditory system (HAS). This means that any noise added to the audio with data embedded in the LSBs can be detected by the HAS, which is dependent on the number of LSBs used. This work presents an improvement of the LSB-based approach, the XORing of LSBs method, with an emphasis on enhanced security. The method achieves this by randomly picking the pairs of bits to be XOR’d while preserving the characteristics of classic LSB methods. To verify the method, it has been implemented on the Cirrus Logic DSP, proving that it may be efficiently used in a real-time audio environment without interfering with HAS sensitivity levels.
An ensemble-based framework for biomedical classification problems

Model selection is an essential step when applying machine learning to classification problems. It is typically carried out by the practitioner who strives to identify the most suitable classifier for a given problem. Given the variety of classifiers available and the difficulty in predicting which one will yield the best performance depending on the characteristics of the problem, this is by no means a simple task. Biomedical problems pose a significant challenge in this regard due to their numerous data intrinsic characteristics that are known to impair classification performance. Given that different classifiers perform well for different biomedical problems, combining them into an ensemble would seem practical. However, the practitioner still needs to determine how to combine them. This paper presents an ensemble-based framework that automates the training and combination of different classifiers in order to relieve practitioners of this burden whilst obtaining highly competitive performance. The effectiveness of the proposed framework was evaluated on several biomedical problems from the literature.
Perspective Chapter: Effective Techniques for Getting and Keeping Students Engaged in a COVID-19 Environment

This chapter offers various techniques that have been used to get and keep students engaged in their learning, given the current state of the world. Based on a wide range of continuing education courses, it will describe ways in which to instruct students, regardless of whether the format is in-person, online, or hybrid. Interactive activities, such as Kahoot!, JeoParody, Family Feud, Quizlet, Quizziz, and Poll Everywhere will be discussed. While recognizing that each student and class is unique, the strategies suggested can set the foundation for promoting an educational environment, stricken by the pandemic, that fully embraces diversity, equity, and inclusion.
Ethical use of data in AI Applications

Artificial Intelligence (AI) provides the mechanisms to enable machines to learn. AI systems which use machine learning, can detect patterns in enormous volumes of data and model complex, interdependent systems to generate outcomes that improve the efficiency of decision making. AI in particular machine learning has been positioned as an important element in contributing to as well as providing decisions in a multitude of industries. The use of machine learning in delivering decisions is based on the data that is used to train the machine learning algorithms. It is imperative that when machine learning applications are being considered that the data being used to train the machine learning algorithms are without bias, and the data is ethically used. This chapter focuses on the ethical use of data in developing machine learning algorithms. Specifically, this chapter will include the examination of AI bias and ethical use of AI, data ethics principles, selecting ethical data for AI applications, AI and data governance, and putting ethical AI applications into practice.
Bird and Micro-Drone Doppler Spectral Width and Classification

This paper reports on the class separability of spectrograms featuring bird and micro-drone targets produced by an L-Band staring radar. Multi-rotor drones with small propellor blades are less likely to show strong micro-Doppler sidebands depending on the range and operating frequency. With this, we were incentivized to measure the separability of the target classes relying only on the body Doppler information captured in the spectrograms. A spectral width feature extraction method was tested using both a set of single drone and bird targets, as well as a larger dataset including spectrograms containing multiple targets and a mixture of classes. These features were employed to inform a simple classifier yielding an 83% classification accuracy in the single target case. The results were then compared to a convolutional neural network baseline that achieved 89% accuracy on the larger, more complex dataset.
Slow motion video sequences database for freezing artifact detection

In this paper, a new video sequence database, called Slow Motion Video Sequences (SMVS), is developed. The developed SMVS database consists of 30 video sequences with very low temporal activities, where every sequence contains a freezing artifact. The performance of two freezing detection algorithms, the Histogram-Based Freezing Artifacts Detection Algorithm (HBFDA) and the Real-Time no-reference Freezing Detection Algorithm (RTFDA) are tested on the developed database. The testing results show the poor performance of the tested algorithms.
Convolutional Neural Network for Joint Communication and Radar Signals Classification

Automatic modulation classification (AMC) plays an important role in the development of cognitive radio and cognitive radar systems. Due to the distinct aims of radar and communication systems, their commonly used modulation techniques may differ significantly. However, AMC in the radar and communication domains can be closely aligned. Considering the widespread applications of deep learning architectures, particularly, convolutional neural networks (CNN) in classification problems, we propose a CNN-based framework for the joint classification of communication and radar signals. In the proposed framework, a CNN is first trained by signals in the in-phase and quadrature domain and then another CNN is trained by using constellation diagrams to differentiate between close modulations. A publicly available benchmark dataset, which consists of different modulation schemes in the communication domain, has been augmented with our simulated linear frequency modulated radar signals under the same noise condition to validate the accuracy of the proposed framework.
Adaptively Managing Reliability of Machine Learning Perception under Changing Operating Conditions

Autonomous systems are deployed in various contexts, which makes the role of the surrounding environment and operational context increasingly vital, e.g., for autonomous driving. To account for these changing operating conditions, an autonomous system must adapt its behavior to maintain safe operation and a high level of autonomy. Machine Learning (ML) components are generally being introduced for perceiving an autonomous system’s environment, but their reliability strongly depends on the actual operating conditions, which are hard to predict. Therefore, we propose a novel approach to learn the influence of the prevalent operating conditions and use this knowledge to optimize reliability of the perception through self-adaptation. Our proposed approach is evaluated in a perception case study for autonomous driving. We demonstrate that our approach is able to improve perception under varying operating conditions, in contrast to the state-of-the-art. Besides the advantage of interpretability, our results show the superior reliability of ML-based perception.
Code Librarian: A Software Package Recommendation System

The use of packaged libraries can significantly shorten the software development life cycle by improving the quality and readability of code. In this paper, we present a recommendation engine called Code Librarian for open source libraries. A candidate library package is recommended for a given context if: 1) it has been frequently used with the imported libraries in the program; 2) it has similar functionality to the imported libraries in the program; 3) it has similar functionality to the developer’s implementation, and 4) it can be used efficiently in the context of the provided code. We apply the state of the art CodeBERT-based model for analysing the context of the source code to deliver relevant library recommendations to users.
Understanding Why and Predicting When Developers Adhere to Code-Quality Standards

Static analysis tools are widely used in software development. While research has focused on improving tool accuracy, evidence at Microsoft suggests that developers often consider some accurately detected warnings not worth fixing: what these tools and developers consider to be true positives differs. Thus, improving tool utilization requires understanding when and why developers fix static-analysis warnings.We conduct a case study of Microsoft’s Banned API Standard used within the company, which describes 195 APIs that can potentially cause vulnerabilities and 142 recommended replacements. We find that developers often (84% of the time) consciously deviate from this standard, specifying their rationale, allowing us to study why and when developers deviate from standards. We then identify 23 factors that correlate with developers using the preferred APIs and build a model that predicts whether the developers would use the preferred or discouraged APIs under different circumstances with 92% accuracy. We also train a model to predict the kind of APIs developers would use in the future based on their past development activity, with 86% accuracy. We outline a series of concrete suggestions static analysis developers can use to prioritize and customize their output, potentially increasing their tools’ usefulness.