Reviewerly/exHarmony huggingface dataset:
FPGA design of FFT based intelligent accelerator with optimized Wallace tree multiplier for image super resolution and quality enhancement

The super-resolution (SR) method has been widely used to improve the resolution and quality of natural images, and it is currently being employed in medical imaging. Recently, many techniques have been introduced by researchers to improve the resolution and quality of the image. However, these approaches generate blurring and jagged edges in images. Hence, this article introduces novel deep learning (DL) with a fast Fourier transform (FFT) based intelligent approach (IA) to improve the quality and resolution of the image. A convolutional neural network (CNN) is emphasized for quality enhancement and resolution to recover lost information during image acquisition. This work undergoes three significant stages: de-noising, segmentation, and resolution improvement of the noisy input image. The double density discrete wavelet transform (DDDWT) technique is introduced to remove the noise and improve the visual image’s quality. Finally, the image resolution and quality enhancement are done using CNN with overlap and add (OA)-FFT based Wallace tree multiplier-red deer optimization (WT-RDO) technique. In the experimental scenario, the quality enhancement and image resolution outcome achieved accuracy of 97 % and 98 %, precision of 97 % and 97 %, sensitivity of 96 % and 98 %, specificity of 96 % and 98 %, PSNR of 30 dB and 26 dB, structural similarity index (SSIM) of 0.86 and 0.85, mean square error (MSE) of 0.01, mean absolute error (MAE) of 0.8, the time complexity of 61, power of 0.08 W, delay of 0.6 ns and throughput of 3.2 MB/s.
Taxonomy and definition of audio augmented reality (AAR): A grounded theory study

AR applications have mostly considered visual augmentations while excluding other modalities. Recent developments in audio augmented reality (AAR) applications have been based on the definitions of visual AR or mixed reality (MR), and thus, AAR technology development has lacked systematic efforts. We investigated the concept of augmented reality through audio to provide a systematic understanding and generate a taxonomy and a definition for AAR. A conference workshop (N = 28), focus groups (N = 18), and expert interviews (N = 6) generated qualitative data regarding the concept of AAR. Grounded Theory (GT) was used to analyze the data and produce a new taxonomy and a definition. The AAR taxonomy consists of three categories – Environment Connected, Goal Directed, and Context Adapted, with three subcategories respectively. The need for a separate taxonomy for AAR is highlighted to aid in the development of AAR applications in a systematic manner. The taxonomy is expected to be used as a heuristic tool that can guide developers to build AAR applications and can be used in evaluating user experience with AAR applications.
Condition Monitoring of Underground Power Cables Via Power-Line Modems and Anomaly Detection

Underground power cables are a widely-deployed class of assets in electric power distribution systems. They are robust in adverse weather conditions and are an aesthetically pleasing alternative to overhead power lines. Unfortunately, the physical observability of underground cables is poor. Current methods for detecting and localizing defects known to progress to failure are cumbersome and expensive, both financially and operationally. In-service cable failures can lead to lengthy power outages and financial losses and can endanger utility staff and the general public. There is thus a need for non-invasive condition monitoring techniques that will allow for early detection of defects likely to progress to a fault, so cost-effective repair or replacement interventions can be planned. In this study, insulation overheating is specifically considered. Thermal damage of XLPE insulation is a prominent damage mechanism for extruded dielectric cable and is thus well studied and understood. A defect detection solution combining powerline modems with semi-supervised anomaly detection for underground cables is proposed. The performance of five well-regarded machine-learning-based anomaly detection algorithms is analyzed through simulation of a real-world 15kV distribution network. The best-performing algorithm studied had a false alarm rate of under 0.1% with no missed alarms across a range of damage scenarios.
High-accuracy vanishing-constraints-based calibration of fringe projection vision sensor

In fringe projection profilometry (FPP), calibration of the vision sensor is crucial to ensure the accuracy of three-dimensional (3D) shape measurement. However, due to correspondence errors of planar calibration target (PCT) 3D points and its image points, and low mapping accuracy between projector pixels and camera pixels, traditional calibration methods based on the PCT sometimes cannot provide reliable and accurate results. Aiming at these problems, a high-accuracy calibration method based on vanishing constraints for fringe projection vision sensor (FPVS) is proposed in this paper. The intrinsic parameters are calibrated using the constraints of vanishing points, the rotation matrix is calibrated using the constraints of vanishing lines, and the optimal calibration parameters are estimated by the objective function established based on the 2D and 3D constraints of feature points. Experimental results demonstrated the performance of our calibration method.
NOVAction23: Addressing the data diversity gap by uniquely generated synthetic sequences for real-world human action recognition

Recognition of human actions using machine learning requires extensive datasets to develop robust models. Nevertheless, obtaining real-world data presents challenges due to the costly and time-consuming process involved. Additionally, existing datasets mostly contain indoor videos due to the challenges of capturing pose data outdoors. Synthetic data have been used to overcome these difficulties, yet the currently available synthetic datasets for human action recognition lack photorealism and diversity in their features. Addressing these shortcomings, we develop the NOVAction engine to generate highly diversified and photorealistic synthetic human action sequences. We use NOVAction to create the NOVAction23 dataset comprising 25,415 human action sequences with corresponding poses and labels (available at https://github.com/celikcan-cglab/NOVAction23). In NOVAction23, the performed motions and viewpoints are varied on the fly through procedural generation, to ensure that, for a given action class, each generated sequence features a distinct motion performed by one of the 1,105 synthetic humans captured from a unique viewpoint. Moreover, each synthetic human is unique in terms of body shape (height and weight), skin tone, gender, hair, facial hair, clothing, shoes and accessories. To further increase data diversity, the motion sequences are rendered under various weather conditions and at different times of day, across three outdoor and two indoor settings. We evaluate NOVAction23 by training three state-of-the-art recognizers on it, in addition to the NTU 120 dataset, and corroborating using real-world videos from YouTube. Our results confirm that the NOVAction23 dataset can improve the performance of state-of-the-art human action recognition.
HASP: Hierarchical Asynchronous Parallelism for Multi-NN Tasks

The rapid development of deep learning has propelled many real-world artificial intelligence applications. Many of these applications integrate multiple neural networks (multi- NN) to cater to various functionalities. There are two challenges of multi-NN acceleration: (1) competition for shared resources becomes a bottleneck, and (2) heterogeneous workloads exhibit remarkably different computing-memory characteristics and various synchronization requirements. Therefore, resource isolation and fine-grained resource allocation for each task are two fundamental requirements for multi-NN computing systems. Although a number of multi-NN acceleration technologies have been explored, few can completely fulfill both of these requirements, especially for mobile scenarios. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">This paper reports a Hierarchical Asynchronous Parallel Model (HASP) to enhance multi-NN performance to meet both requirements. HASP can be implemented on a multicore processor that adopts Multiple Instruction Multiple Data (MIMD) or Single Instruction Multiple Thread (SIMT) architectures, with minor adaptive modification needed. Further, a prototype chip is developed to validate the hardware effectiveness of this design. A corresponding mapping strategy is also developed, allowing the proposed architecture to simultaneously promote resource utilization and throughput. With the same workload, the prototype chip demonstrates 3.62×, and 3.51× higher throughput over Planaria and 8.68×, 2.61× over Jetson AGX Orin for MobileNet-V1 and ResNet50, respectively.
Entropy-Oriented Domain Adaptation for Intelligent Diagnosis of Rotating Machinery

To cater to fault diagnosis of rotating machinery under complex working conditions, unsupervised domain adaptation technology has been widely explored and applied. Existing methods mainly reduce domain bias in two ways, including metric learning and discriminator-based adversarial learning. Different from these technologies, in this work, we only resort to entropy optimization strategies and develop a novel entropy-oriented domain adaptation (EODA) model for intelligent diagnosis of rotating machinery. Specifically, a convolutional network with a cosine-distance classifier is introduced to construct the model framework, which can reduce intraclass variation and make the output more confident. In addition, negentropy-guided prediction diversity optimization and minimax entropy game-guided prototype-feature alignment are co-designed to realize domain adaptation. Extensive experiments based on two different mechanical systems are used to validate our method. Comprehensive results and discussions demonstrate that our EODA can achieve compelling performance.
Artificial Intelligence-Driven Mammography-Based Future Breast Cancer Risk Prediction: A Systematic Review

To summarize the literature regarding the performance of mammography-image based artificial intelligence (AI) algorithms, with and without additional clinical data, for future breast cancer risk prediction.A systematic literature review was performed using six databases (medRixiv, bioRxiv, Embase, Engineer Village, IEEE Xplore, and PubMed) from 2012 through September 30, 2022. Studies were included if they used real-world screening mammography examinations to validate AI algorithms for future risk prediction based on images alone or in combination with clinical risk factors. The quality of studies was assessed, and predictive accuracy was recorded as the area under the receiver operating characteristic curve (AUC).Sixteen studies met inclusion and exclusion criteria, of which 14 studies provided AUC values. The median AUC performance of AI image-only models was 0.72 (range 0.62-0.90) compared with 0.61 for breast density or clinical risk factor-based tools (range 0.54-0.69). Of the seven studies that compared AI image-only performance directly to combined image + clinical risk factor performance, six demonstrated no significant improvement, and one study demonstrated increased improvement.Early efforts for predicting future breast cancer risk based on mammography images alone demonstrate comparable or better accuracy to traditional risk tools with little or no improvement when adding clinical risk factor data. Transitioning from clinical risk factor-based to AI image-based risk models may lead to more accurate, personalized risk-based screening approaches.
Convergence rates for shallow neural networks learned by gradient descent

In this paper we analyze the L2 error of neural network regression estimates with one hidden layer. Under the assumption that the Fourier transform of the regression function decays suitably fast, we show that an estimate, where all initial weights are chosen according to proper uniform distributions and where the weights are learned by gradient descent, achieves a rate of convergence of 1∕n (up to a logarithmic factor). Our statistical analysis implies that the key aspect behind this result is the proper choice of the initial inner weights and the adjustment of the outer weights via gradient descent. This indicates that we can also simply use linear least squares to choose the outer weights. We prove a corresponding theoretical result and compare our new linear least squares neural network estimate with standard neural network estimates via simulated data. Our simulations show that our theoretical considerations lead to an estimate with an improved performance in many cases.
Risk-Averse Trajectory Optimization via Sample Average Approximation

Trajectory optimization under uncertainty underpins a wide range of applications in robotics.However, existing methods are limited in terms of reasoning about sources of epistemic and aleatoric uncertainty, space and time correlations, nonlinear dynamics, and non-convex constraints.In this work, we first introduce a continuous-time planning formulation with an average-value-at-risk constraint over the entire planning horizon.Then, we propose a sample-based approximation that unlocks an efficient and general-purpose algorithm for risk-averse trajectory optimization.We prove that the method is asymptotically optimal and derive finite-sample error bounds.Simulations demonstrate the high speed and reliability of the approach on problems with stochasticity in nonlinear dynamics, obstacle fields, interactions, and terrain parameters.
Structural asymmetric convolution for wireframe parsing

Simultaneously extracting junctions and their corresponding line segments from images presents a promising approach to structural environment cognition. However, conventional methods employ square convolution for line feature extraction, resulting in the exclusion of long-range dependencies and the generation of suboptimal wireframe predictions. In this paper, we introduce an efficient and concise parsing method named Structural Asymmetric Convolution-based Wireframe Parser (SACWP). Taking advantage of the inherent similarities between structural asymmetric convolution and the predominant distribution of line segments in man-made environments, we propose a Structural Asymmetric Convolution module (SAC) that captures long-range contextual features while efficiently filtering out irrelevant information from neighboring pixels. Additionally, we introduce a feature aggregation module based on dilated convolution (DCFA) to seamlessly integrate contextual information from multiple receptive fields. We thoroughly evaluate our approach on the Wireframe and YorkUrban datasets, achieving preferable results of 69.3% and 29.7% msAP respectively. On the other hand, the promising results adequately demonstrate the effectiveness of SACWP to Wireframe Parsing task.
A novel privacy-preserving graph convolutional network via secure matrix multiplication

Graph convolutional network (GCN) is one of the most representative methods in the realm of graph neural networks (GNNs). In the convolution process, GCN combines the structural information of the networks with the features of nodes. In practice, the structure information of the networks and the features of nodes may be controlled by different parties, which can render GCN ineffective when information sharing is constrained by privacy concerns or licensing issues. Therefore, it is of significant importance to design an effective and secure scheme for GCN that can collaboratively merge information from both parties while safeguarding their sensitive data. In this paper, we introduce S-GCN (i.e., Secure GCN Scheme), which employs secure matrix multiplication (SMM) to compute the product of two matrices in a privacy-preserving manner. The S-GCN scheme requires frequent utilization of SMM to merge information from both parties, resulting in high time and space complexity. To address this issue, we introduce SF-GCN (i.e., Secure and Fast GCN Scheme), which minimizes the use of SMM. Additionally, both S-GCN and SF-GCN may be susceptible to privacy breaches when dealing with dense networks. Hence, we further enhance security by introducing differential privacy through the Laplacian mechanism. Experimental results demonstrate that the proposed schemes do not significantly reduce accuracy in downstream tasks, and, more importantly, effectively protect the privacy information of both parties.
Comparison of current quantum devices for quantum computing of Heisenberg spin chain dynamics

We simulate the time-dependent dynamics of a chain of three interacting spins described by the Heisenberg XXX Hamiltonian using two types of different quantum computers: Quantinuum H1-1, a trapped-ion quantum computer, and ibm_prague, a superconducting qubit type quantum computer. The time evolution of the wave function is accomplished by the Suzuki–Trotter approximation. We find that very accurate results are obtained using H1-1 after applying error mitigation by renormalization of the time-dependent spin-state populations as well as using ibm_prague after renormalization, dynamical decoupling, Pauli twirling, and measurement error mitigation of the populations.
Multiscale variational autoencoder regressor for production prediction and energy saving of industrial processes

As modern industries increase in scale and integration, the industrial process data show more complex dynamic time variability. To solve the dynamic time-varying problem of industrial process data, a novel multiscale variational autoencoder (MSVAE) based Regressor (REG) (MSVAE-REG) is proposed for production prediction and energy saving. The encoder of the MSVAE is constructed by multiscale convolutional neural network (MSCNN) to extract the multiscale temporal dynamic features, including the overall and local trend features in the Gaussian latent space. Then, the decoder of the MSVAE recovers the input data to enhance the dynamic adaptability of multiscale features. Moreover, the REG utilizes the gated recurrent unit (GRU) to build a dynamic relationship between multiscale temporal dynamic features and the predicted output. Finally, the proposed model is verified on the propylene and ethylene production datasets. Compared with the back propagation (BP) network, the extreme learning machine (ELM), the radial basis function (RBF), the convolutional neural network (CNN) and the variational autoencoder (VAE), the MSVAE-REG achieves state-of-the-art results, with the prediction accuracy respectively reaching about 95% and 96%, providing a new judgment basis for optimizing the production process and saving energy.
Monocular depth estimation using self-supervised learning with more effective geometric constraints

Self-supervised learning-based depth estimation from monocular videos is a challenging yet promising way for 3D environment perception. Existing methods that use photometric consistency as supervision are often fragile in the case of textureless environments or illumination variations. In this paper, we adopt geometric constraints to improve the reliability and accuracy of monocular depth estimation. We introduce an equation derived from the two-view imaging geometry, from which we also develop a novel geometric loss function that is shown to be more effective than the epipolar constraint in self-supervised learning. In addition, we design a depth reconstruction loss by explicitly aligning the depth scale of the consecutive estimated depth maps to keep the depth scale consistent. Finally, a new self-supervised learning framework is formulated by incorporating the proposed geometric constraints into depth, pose, and optical flow estimation models. Extensive experiments on KITTI, Make3D, and NYU Depth v2 datasets show that, by adding our geometric constraints, the depth estimation model compares favorably with state-of-the-art self-supervised learning methods, and each sub-task has a significant improvement.
Dual attention and channel transformer based generative adversarial network for restoration of the damaged artwork

Artworks are treasures of valuable cultural and historical heritage. Artworks get damaged due to environmental and other factors. The artificial intelligence-based restoration of digitized artwork images can guide the artists in physically restoring the damaged artworks. Previous methods have not been able to restore artwork images well. This paper proposes a dual (spatial and channel) attention and channel transformer-based generative adversarial network to restore damaged artwork images digitally. The proposed generative adversarial network has spatial and channel attention layers in the encoder part of the generator and a channel transformer between skip connections from the encoder to the decoder part of the generator. Spatial and channel attention helps learn inter-spatial and inter-channel global relationships among image features. Channel transformer ensures multiscale feature fusion and reduces the semantic gap between encoder and decoder layer features. Moreover, the proposed network has been trained using a linear combination of perceptual, adversarial, and structured similarity index measure loss, which helps better train the network. Further, the proposed network has been validated on two different datasets, and the results indicate that the proposed method outperforms state-of-the-art artwork restoration methods.
TransPV: Refining photovoltaic panel detection accuracy through a vision transformer-based deep learning model

The increasing need to develop renewable energy sources to combat climate change has led to a significant rise in demand for photovoltaic (PV) installations. Consequently, accurately detecting and estimating the capacity and potential for electricity generation of these installed PV systems has become crucial for effective energy management. However, due to the limitation of localized mechanism in convolution operations, traditional FCN-based methods face challenges in capturing global dependencies among remote sensing image patches. As a result, they struggle to model long-range interactions between different PV panels with diverse structures, including varying size, shape, and texture, particularly in distributed residential areas. To tackle the challenge of modeling PV panels with diverse structures, we propose a coupled U-Net and Vision Transformer model named TransPV for refining PV semantic segmentation. Specifically, Mix Transformer block is incorporated in the encoder to enhance the modeling of global context, while the U-Shaped structure enables the combination of multi-level features, resulting in enriched feature representation, which helps the model better understand objects in the images, regardless of their shape, size, and texture. In addition, we implement the PointRend module in the decoder to obtain finer segmentation boundary details, and utilize a novel refiner loss function during the training process to alleviate the problem of extremely unbalanced samples. The experimental results from the Heilbronn datasets demonstrate the remarkable performance of our proposed TransPV model in addressing intra-class diversity of PV structures, surpassing previous state-of-the-art methods with an impressive IoU and accuracy of 0.802 and 0.876, respectively. Furthermore, our model exhibits high generalization capability on the BDPV dataset with IoU of 0.745. The obtained results highlight the superiority of TransPV in improving accuracy and addressing diverse structure issues in PV segmentation, which provides valuable insights for optimizing the efficiency and sustainability of renewable energy.