Reviewerly/exHarmony huggingface dataset:
ConvS2S-VC: Fully Convolutional Sequence-to-Sequence Voice Conversion

This article proposes a voice conversion (VC) method using sequence-to-sequence (seq2seq or S2S) learning, which flexibly converts not only the voice characteristics but also the pitch contour and duration of input speech. The proposed method, called ConvS2S-VC, has three key features. First, it uses a model with a fully convolutional architecture. This is particularly advantageous in that it is suitable for parallel computations using GPUs. It is also beneficial since it enables effective normalization techniques such as batch normalization to be used for all the hidden layers in the networks. Second, it achieves many-to-many conversion by simultaneously learning mappings among multiple speakers using only a single model instead of separately learning mappings between each speaker pair using a different model. This enables the model to fully utilize available training data collected from multiple speakers by capturing common latent features that can be shared across different speakers. Owing to this structure, our model works reasonably well even without source speaker information, thus making it able to handle any-to-many conversion tasks. Third, we introduce a mechanism, called the conditional batch normalization that switches batch normalization layers in accordance with the target speaker. This particular mechanism has been found to be extremely effective for our many-to-many conversion model. We conducted speaker identity conversion experiments and found that ConvS2S-VC obtained higher sound quality and speaker similarity than baseline methods. We also found from audio examples that it could perform well in various tasks including emotional expression conversion, electrolaryngeal speech enhancement, and English accent conversion.
Leveraging Semisupervised Hierarchical Stacking Temporal Convolutional Network for Anomaly Detection in IoT Communication

The rapid development of the Internet of Things (IoT) accumulates a large number of communication records, which are utilized for anomaly detection in IoT communication. However, only a small part of these records can be labeled, which increases the difficulty in anomaly detection. This article proposes a semisupervised hierarchical stacking temporal convolutional network (HS-TCN), which is the first semisupervised model for anomaly detection in IoT communication, and it can train unlabeled data based on a small number of labeled data. Furthermore, HS-TCN fully considers the features of streaming data in IoT communication and can weed out uncertain records. Finally, the experimental results demonstrate that HS-TCN promotes the performance of anomaly detection and attains better efficiency.
Focus on Defocus: Bridging the Synthetic to Real Domain Gap for Depth Estimation

Data-driven depth estimation methods struggle with the generalization outside their training scenes due to the immense variability of the real-world scenes. This problem can be partially addressed by utilising synthetically generated images, but closing the synthetic-real domain gap is far from trivial. In this paper, we tackle this issue by using domain invariant defocus blur as direct supervision. We leverage defocus cues by using a permutation invariant convolutional neural network that encourages the network to learn from the differences between images with a different point of focus. Our proposed network uses the defocus map as an intermediate supervisory signal. We are able to train our model completely on synthetic data and directly apply it to a wide range of real-world images. We evaluate our model on synthetic and real datasets, showing compelling generalization results and state-of-the-art depth prediction. The dataset and code are available at https://github.com/dvl-tum/defocus-net.
Cybersecurity vulnerability management: A conceptual ontology and cyber intelligence alert system

Effective vulnerability management requires the integration of vulnerability information available on multiple sources, including social media. The information could be used to inform common users about impending vulnerabilities and countermeasures. First, we present the Cybersecurity Vulnerability Ontology (CVO), a conceptual model for formal knowledge representation of the vulnerability management domain. Second, we utilize the CVO to design a Cyber Intelligence Alert (CIA) system that issues cyber alerts about vulnerabilities and countermeasures. We rigorously evaluated the CVO as well as the accuracy, performance, and usefulness of the CIA system. Key contributions of this study to research and practice are discussed.
Decoupled Representation Learning for Skeleton-Based Gesture Recognition

Skeleton-based gesture recognition is very challenging, as the high-level information in gesture is expressed by a sequence of complexly composite motions. Previous works often learn all the motions with a single model. In this paper, we propose to decouple the gesture into hand posture variations and hand movements, which are then modeled separately. For the former, the skeleton sequence is embedded into a 3D hand posture evolution volume (HPEV) to represent fine-grained posture variations. For the latter, the shifts of hand center and fingertips are arranged as a 2D hand movement map (HMM) to capture holistic movements. To learn from the two inhomogeneous representations for gesture recognition, we propose an end-to-end two-stream network. The HPEV stream integrates both spatial layout and temporal evolution information of hand postures by a dedicated 3D CNN, while the HMM stream develops an efficient 2D CNN to extract hand movement features. Eventually, the predictions of the two streams are aggregated with high efficiency. Extensive experiments on SHREC'17 Track, DHG-14/28 and FPHA datasets demonstrate that our method is competitive with the state-of-the-art.
Time Matters

Incorporating temporal information into recommender systems has recently attracted increasing attention from both the industrial and academic research communities. Existing methods mostly reduce the temporal information of behaviors to behavior sequences for subsequently RNN-based modeling. In such a simple manner, crucial time-related signals have been largely neglected. This paper aims to systematically investigate the effects of the temporal information in sequential recommendations. In particular, we firstly discover two elementary temporal patterns of user behaviors: "absolute time patterns'' and "relative time patterns'', where the former highlights user time-sensitive behaviors, e.g., people may frequently interact with specific products at certain time point, and the latter indicates how time interval influences the relationship between two actions. For seamlessly incorporating these information into a unified model, we devise a neural architecture that jointly learns those temporal patterns to model user dynamic preferences. Extensive experiments on real-world datasets demonstrate the superiority of our model, comparing with the state-of-the-arts.
A simulation model of the irradiation and energy yield of large bifacial photovoltaic plants

Open questions remain regarding the use of bifacial photovoltaic (PV) modules, such as how to link the energy-yield modelling of large bifacial PV plants with further on-site measurements campaigns, the effect of the non-uniformity of the rear irradiance, as well as the convenience of varying the geometry of the plants to enhance the bifacial contribution. A simplified two-dimensional view factor model for bifacial PV plants is presented in this paper, as well as a specific definition of performance ratio (PR) for this type of installations. This model has been implemented in SISIFO --an open simulation tool --for static structures and horizontal single-axis trackers. Yearly energy yield has been simulated for two representative static and tracker PV plants located in Madrid with an albedo of 0.3 and a bifacial power coefficient of 0.7, resulting in yield increases driven by rear irradiation of 8% and 7%, respectively. In general, the influence of varying structure height is lower than that of row-to-row spacing. The model also reveals a non-homogeneous rear irradiance. This question of non-uniformity of rear irradiance is analyzed in light of experimental 10-month observations in a 6.6 kWp PV system. The average of the rear irradiance observed at the center and at the two borders of the PV array is an adequate predictor for DC power estimations. A pioneering chapter of bifacial PV modules industrialization that took place between 1984 and 1989 in Spain, and probably the first application of view factor to model bifacial PV modules, are also presented.
Efficient hyperparameter optimization through model-based reinforcement learning

Hyperparameter tuning is critical for the performance of machine learning algorithms. However, a noticeable limitation is the high computational cost of algorithm evaluation for complex models or for large datasets, which makes the tuning process highly inefficient. In this paper, we propose a novel model-based method for efficient hyperparameter optimization. Firstly, we frame this optimization process as a reinforcement learning problem and then employ an agent to tune hyperparameters sequentially. In addition, a model that learns how to evaluate an algorithm is used to speed up the training. However, model inaccuracy is further exacerbated by long-term use, resulting in collapse performance. We propose a novel method for controlling the model use by measuring the impact of the model on the policy and limiting it to a proper range. Thus, the horizon of the model use can be dynamically adjusted. We apply the proposed method to tune the hyperparameters of the extreme gradient boosting and convolutional neural networks on 101 tasks. The experimental results verify that the proposed method achieves the highest accuracy on 86.1% of the tasks, compared with other state-of-the-art methods and the average ranking of runtime is significant lower than all methods by using the predictive model.
Observing 360-degree classroom videos – Effects of video type on presence, emotions, workload, classroom observations, and ratings of teaching quality

The use of classroom videos has become an increasingly popular topic in teacher education and educational research. Usually, video-based classroom observation is done using a "normal" 4:3 or 16:9 video recording from a given camera angle. By contrast, 360-degree videos do not predefine a specific camera angle during the recording, and the observer is able to choose the viewing direction while watching the video. Furthermore, research on 360-degree videos from other disciplines indicates that such videos might elicit more presence and emotions than traditional 16:9 videos, raising the question whether observations of teaching differ with respect to the type of video. Thus, the aim of the present experimental study was to explore whether perceived presence, emotions, workload, classroom observations, and ratings of teaching quality differ when student teachers are shown a 360-degree classroom video and when they are shown a traditional 16:9 classroom video. Fiftynine student teachers were randomly assigned either to watch a 360-degree video or a traditional 16:9 video. After watching the video, they indicated their perceived presence, emotions, workload, and ratings of teaching quality. They also analysed the video by describing and interpreting the relevant events they had noticed. The results showed that the 360-degree video elicited a higher degree of presence. However, there were no significant differences between the two video types with respect to perceived emotions, workload, observer ratings, or noticed events. We discuss the findings in terms of using classroom videos for research purposes and also for teacher education.
Efficient Step-Merged Quantum Imaginary Time Evolution Algorithm for Quantum Chemistry

We develop a resource-efficient step-merged quantum imaginary time evolution approach (smQITE) to solve for the ground state of a Hamiltonian on quantum computers. This heuristic method features a fixed shallow quantum circuit depth along the state evolution path. We use this algorithm to determine the binding energy curves of a set of molecules, including H2, H4, H6, LiH, HF, H2O, and BeH2, and find highly accurate results. The required quantum resources of smQITE calculations can be further reduced by adopting the circuit form of the variational quantum eigensolver (VQE) technique, such as the unitary coupled cluster ansatz. We demonstrate that smQITE achieves a similar computational accuracy as VQE at the same fixed-circuit ansatz, without requiring a generally complicated high-dimensional nonconvex optimization. Finally, smQITE calculations are carried out on Rigetti quantum processing units, demonstrating that the approach is readily applicable on current noisy intermediate-scale quantum devices.
Nonlinear systems modelling based on self-organizing fuzzy neural network with hierarchical pruning scheme

In this paper, a self-organizing fuzzy neural network with hierarchical pruning scheme (SOFNN-HPS) is proposed for nonlinear systems modelling in industrial processes. In SOFNN-HPS, to strike the optimal balance between system accuracy and network complexity, an online self-organizing scheme for identifying the structure and parameters of the network simultaneously is developed. First, to enhance the characterization ability of the fuzzy rules for nonlinear systems, the asymmetric Gaussian functions that can partition the input space more flexibly are introduced as membership functions. Second, a hierarchical pruning scheme, which is designed by rule density and rule significance, is used to delete the redundant fuzzy rules while using the geometric growing criteria to generate fuzzy rules automatically, which can avoid the requirement of pre-setting the pruning threshold and prevent the mistaken deletion of significant rules. Third, an adaptive allocation strategy is adopted to set the antecedent parameters of the fuzzy rules in the learning process, which can not only adjust the region of generalized ellipsoidal basis functions for better local approximation, but also balance the accuracy of the system and the interpretability of the rule base obtained. Finally, to speed up the convergence of the estimation error, a modified recursive least square algorithm is used to update the consequent parameters of the resulting fuzzy rules online. In addition, the convergence proofs of the estimation error and the network linear parameters of SOFNN-HPS are given, and they are helpful in successfully applying the SOFNN-HPS in practical engineering. To verify the effectiveness of SOFNN-HPS, two benchmark test problems and a key water quality parameter prediction experiment in the wastewater treatment process are examined. The simulation results demonstrate that the proposed SOFNN-HPS algorithm can obtain a self-organizing fuzzy neural network with compact structure and powerful generalization performance. The source codes of SOFNN-HPS and other competitors can be downloaded from https://github.com/hyitzhb/SOFNN-HPS.
Benchmarking Underwater Image Enhancement and Restoration, and Beyond

Image enhancement and restoration is among the most investigated topics in the field of underwater machine vision. The objective image quality assessment is a fundamental part of optimizing underwater enhancement and restoration technologies. However, most no-reference (NR) metrics are not specifically designed for underwater image quality assessment. Moreover, since the reference (undegraded) images are not available in underwater scenes, the classical full-reference (FR) metrics cannot be used to evaluate underwater image enhancement and restoration methods. In this paper, we first design an underwater image synthesis algorithm (UISA), in which depending on the real-world underwater image, we can produce a synthetic underwater image from an outdoor ground-truth image. Based on this strategy, we establish a new large-scale benchmark that contains ground-truth images and synthetic underwater images of the same scene, called synthetic underwater image dataset (SUID). Our SUID is constructed on the basis of the underwater image formation model (IFM) and characteristics of underwater optical propagation, possessing solid reliability and feasibility. The proposed SUID creates possibility for a FR evaluation of existing technologies for underwater image enhancement and restoration, which is illustrated by performing extensive experiments and quantitative analysis. The SUID is available online at: http://dx.doi.org/10.21227/agdr-y109.
Blockchain and Trust for Secure, End-User-Based and Decentralized IoT Service Provision

Building trust relationships between different decentralized entities in the IoT ecosystem is essential. Hereof, the combination of blockchain technology and trust evaluation techniques is recently considered as an efficient measure. However, both technologies within the IoT are still facing some limitations which are addressed in this research. First, this publication reviews various blockchain-based trust approaches and depicts their strengths and limitations regarding their usage in decentralized IoT communities. Then, an optimized trust model with a multi-layer adaptive and trust-based weighting system is proposed. Additionally, different trust metric parameters and their mathematical models used for trust evaluation are presented. Moreover, this publication presents a novel approach for incentivization processes in the IoT marketplace using control loops and smart contracts. Thereby, participants are motivated to continuously improve their behavior. Finally, the proposed trust model is proved to be reliable. The experimental results conducted from different scenarios show that the presented approach provides more resiliency against various attacks than existing ones.
A Provably Secure and Lightweight Identity-Based Two-Party Authenticated Key Agreement Protocol for IIoT Environments

With the significant development of the Internet, Internet of Things (IoT) has become an emerging technology in many industries. To support security and privacy in the Industrial IoT environment, a user may interact with another user on the Internet to share confidential information, which requires an authenticated communication channel. To meet this demand, in this article the authors developed an identity-based two-party authenticated key agreement (ID-2PAKA) protocol that allows two users to communicate securely and share sensitive data across IoT-enabled regions. Similar protocols found in the literature either proven to be insecure or carry the burden of high communication and computational costs. The proposed ID-2PAKA protocol is analyzed in the random oracle model to achieve provable security based on the hardness assumptions of computational Diffie-Hellman and bilinear Diffie-Hellman problems. The performance analysis of the proposed ID-2PAKA protocol is performed using the pairing-based cryptography library. The comparative results from the perspective of the computation and communication costs against the competing protocols showed that the proposed ID-2PAKA protocol is secure and efficient.
Knowledge-guided synthetic medical image adversarial augmentation for ultrasonography thyroid nodule classification

Background and objective: Image classification is an important task in many medical applications. Methods based on deep learning have made great achievements in the computer vision domain. However, they typically rely on large-scale datasets which are annotated. How to obtain such great datasets is still a serious problem in medical domain. Methods: In this paper, we propose a knowledge-guided adversarial augmentation method for synthesizing medical images. First, we design Term and Image Encoders to extract domain knowledge from radiologists, then we use domain knowledge as novel condition to constrain the Auxiliary Classifier Generative Adversarial Network (ACGAN) framework for the synthesis of high-quality thyroid nodule images. Finally, we demonstrate our method on the task of classifying ultrasonography thyroid nodule. Our method can make effective use of the high-quality diagnostic experience of advanced radiologists. In addition, we creatively choose to extract domain knowledge from standardized terms rather than ultrasound images. Results: Our novel method is demonstrated on a limited dataset of 1937 clinical thyroid ultrasound images and corresponding standardized terms. The accuracy of the proposed model for thyroid nodules is 91.46%, the sensitivity is 90.63%, the specificity is 92.65%, and the AUC is 95.32%, which is better than the current classification methods for thyroid nodules. The experimental results show the model has better generalization and robustness. Conclusions: We believe that the proposed method can alleviate the problem of insufficient data in the medical domain, and other medical problems can benefit from using synthetic augmentation.
AaYusH: A Smart Contract-Based Telesurgery System for Healthcare 4.0

Telesurgery (TS) with 5G-enabled Tactile Internet (TI) has enormous potential to deliver real-time ultra-responsive surgical services remotely with high quality and accuracy. It is quite beneficial for society in the prospect of highly precise surgical diagnosis. However, the existing TS systems have security, privacy, latency, and blockchain (BC) storage cost issues, which restricts its applicability in surgical procedures across the world in the near future. To mitigate the above-mentioned issues, in this paper, we propose an approach named AaYusH (Ethereum smart contract (ESC) and IPFS-based TS system). The security and privacy issues in AaYusH can be resolved through ESC, whereas storage cost issues with the InterPlanetary File System (IPFS) protocol. Moreover, we present a real-time SC written in Solidity and deployed in Truffle suite. We test the security bugs of AaYusH in MyThril open-source tool and detect no issues. Finally, we evaluate the performance of AaYusH in context to latency and data storage cost, and it outperforms as compared to the traditional telesurgery system.
WideSegNeXt: Semantic Image Segmentation Using Wide Residual Network and NeXt Dilated Unit

Semantic segmentation is widely applied in autonomous driving, in robotic picking, and for medical purposes. Due to the breakthrough of deep learning in recent years, the fully convolutional network (FCN)-based method has become the de facto standard in semantic segmentation. However, the simple FCN has difficulty in capturing global context information, since the local receptive field is small. Furthermore, there is a problem of low image resolution because of the existence of the pooling layer. In this paper, we address the shortcomings of the FCN by proposing a new architecture called WideSegNeXt, which captures the image context on various spatial scales and is effective in identifying small objects. In addition, there is little loss of position information, since there are no pooling layers in the structure. The proposed method achieves a mean intersection over union (MIoU) of 72.5% and a global accuracy (GA) of 92.4% on the CamVid dataset and achieves higher performance than previous methods without additional input datasets.