Reviewerly/exHarmony huggingface dataset:
Investigation of Ensemble features of Self-Supervised Pretrained Models for Automatic Speech Recognition

Self-supervised learning (SSL) based models have been shown to generate powerful representations that can be used to improve the performance of downstream speech tasks. Several state-of-the-art SSL models are available, and each of these models optimizes a different loss which gives rise to the possibility of their features being complementary. This paper proposes using an ensemble of such SSL representations and models, which exploits the complementary nature of the features extracted by the various pretrained models. We hypothesize that this results in a richer feature representation and shows results for the ASR downstream task. To this end, we use three SSL models that have shown excellent results on ASR tasks, namely HuBERT, Wav2vec2.0, and WaveLM. We explore the ensemble of models fine-tuned for the ASR task and the ensemble of features using the embeddings obtained from the pre-trained models for a downstream ASR task. We get improved performance over individual models and pre-trained features using Librispeech(100h) and WSJ dataset for the downstream tasks.
Ensemble Interval Prediction for Solar Photovoltaic Power Generation

In recent years, solar photovoltaic power generation has emerged as an essential means of energy supply. The prediction of its active power is not only conducive to cost saving but can also promote the development of solar power generation industry. However, it is challenging to obtain an accurate and high-quality interval prediction of active power. Based on the data set of desert knowledge in the Australia solar center in Australia, firstly, we have compared twelve interval prediction methods based on machine learning. Secondly, six ensemble methods, namely Ensemble-Mean, Ensemble-Median (Ensemble-Med), Ensemble-Envelop (Ensemble-En), Ensemble-Probability averaging of endpoints and simple averaging of midpoints (Ensemble-PM), Ensemble-Exterior trimming (Ensemble-TE), and Ensemble-Interior trimming (Ensemble-TI) are used to combine forecast intervals. The result indicates that Ensemble-TE is the best method. Additionally, compared to other methods, Ensemble-TE ensures the prediction interval coverage probability for confidence levels of 95%, 90%, 85%, and 80% as 0.960, 0.920, 0.873, and 0.824, respectively, using 15-min level data. Meanwhile, the narrower prediction interval normalized averaged width is calculated for the same confidence levels as 0.066, 0.045, 0.035, and 0.028, respectively. In addition, higher Winkler score and smaller coverage width-based criterion are obtained, representing high-quality intervals. We have calculated smaller mean prediction interval center deviation, which is approximately 0.044. Thus, the above demonstrates that this study obtains the prediction interval with better performance compared to other existing methods.
Hformer: Hybrid convolutional neural network transformer network for fringe order prediction in phase unwrapping of fringe projection

Deep learning based on convolutional neural network (CNN) has attracted more and more attention in phase unwrapping of fringe projection three-dimensional (3D) measurement. However, due to the inherent limitations of convolutional operator, it is difficult to accurately determine the fringe order in wrapped phase patterns that rely on continuity and globality. To attack this problem, in this paper we develop a hybrid CNN-transformer model (Hformer) dedicated to phase unwrapping via fringe order prediction. The proposed Hformer model has a hybrid CNN-transformer architecture that is mainly composed of backbone, encoder, and decoder to take advantage of both CNN and transformer. Backbone is used as a wrapped phase pattern feature extractor. Encoder and decoder with cross attention are designed to enhance global dependency for the fringe order prediction. Experimental results show that the proposed Hformer model achieves better performance in fringe order prediction compared with the CNN models such as U-Net and DCNN. Our work opens an alternative way to the CNN-dominated deep learning phase unwrapping of fringe projection 3D measurement.
Service Constraint NCBQ trust orient secure transmission with IoT devices for improved data security in cloud using blockchain

Problem of secure transmission in cloud systems has been well studied; there exist number of approaches discussed around the problem. Number of threats has been identified in cloud which targets the services and data behind them. To mitigate the threats, different approaches are discussed in literature. The most approaches uses behavior of user in measuring trust, and some of them uses data level features in measuring trust of user. Also, various data encryption schemes are discussed which suffer with poor performance in security achievement. To handle this issue, an efficient Service Constraint NCBQ (Network Condition and Behavior Quality) Trust orient secure transmission protocol (SCNCQB-TSTP) is presented in this article. The proposed model involves in secure transmission by analyzing various factors of network, user behavior and quality of service. The method discovers set of routes to reach the service point and fetches several information about the transmission route like traffic, transmission frequency, bandwidth, IoT devices, hop count, throughput, and bandwidth utilization. According to the above information fetched about different routes, the method analyzes the trust of each route in three ways like Network Support Analysis, Behavior Support Analysis and QoS-Support Analysis. Based on the result of different analysis, the method estimates the value of Secure Transmission Score (STS). According to the value of STS, an optimal route has been selected to perform data transmission. Similarly, the method adapts Service Constraint Blockchain Model (SCBM) towards data security, which classifies the services in various levels. According to the service claimed by the user, the service data has been encrypted with specific class of scheme and key. Further, encrypted data has been placed is adapted to block chain to perform data transmission. The proposed SCNCQB-TSTP model improves the performance of secure transmission and data security.
Fission: Autonomous, Scalable Sharding for IoT Blockchain

IoT blockchain suffers heavy performance issues because of the massive transactions generated by various IoT nodes. By dividing nodes into different shards, sharding can produce blocks in parallel and hence improve the throughput of the blockchain system. Unlike the traditional blockchain system, IoT blockchain mainly consists of smart devices and the transactions are usually generated from the real world, such as the sensor data, photos taken by cameras, and so on. In IoT blockchain, closer nodes usually share a lower network latency and the transactions they generate are more related. Therefore, location-based sharding is an effective approach to improve the performance of IoT blockchain. Traditionally, IoT nodes are di-vided into different shards based on geographical locations or the connected edge server. However, the key challenge of sharding in IoT blockchain is how to guarantee the equality of shards division as to the unpredictable distribution and the dynamic behavior of IoT nodes. On one hand, shards can not be pre-divided because we can not predict the number or the distribution of the IoT nodes. On the other hand, nodes continuously joining or quitting shards will also break the equality of the shards division. In this paper, we propose Fission, a sharding mechanism designed for IoT blockchain. Fission divide shards based on the Voronoi diagram without any preknowledge about the nodes distribution, and support dynamic, autonomous sharding adjustment based on distributed Delaunay Triangulation. In addition, Fission uses a new diffusion-based consensus algorithm to achieve the linear scalability of throughput. The experimental results show that Fission can construct and adjust shards at a very low cost and can execute in a decentralized manner. The throughput can reach 1900tps in 500 nodes with only 5M bps bandwidth, and can scale linearly as the nodes increase.
An Effective Coaxiality Measurement for Twist Drill Based on Line Structured Light Sensor

Aiming at the accurate and effective coaxiality measurement of twist drills with irregular surface, an optical measurement mechanism is proposed in this paper. First, A high-precision rotation instrument based on four core units is designed, which can obtain 3D point cloud data of twist drills at all angles. Second, in the data processing stage, an improved robust Gaussian Mixture Model is proposed for accurate and rapid blade back segmentation. To improve the measurement efficiency, a rapid reconstruction method of twist drill axis based on orthogonal synthesis is proposed, which can rapidly locate the maximum deviation between the actual axis and the reference by using the extracted blade back data. Finally, by calculating the maximum radial Euclidean distance from the benchmark, the coaxiality error of the twist drill is obtained. Comparing with other measurement methods, the experimental results show that our proposed method performs well with high efficiency of less than 3 s/pc, and the average measurement error is about 0.020 mm. The experimental results and uncertainty analysis show that the proposed method is effective and automatic, and can be applied to the coaxiality measurement of twist drills of various specifications.
From PHY to QoE: A Parameterized Framework Design

The rapid development of 5G communication technology has given birth to various real-time broadband communication services, such as augmented reality (AR), virtual reality (VR) and cloud games. Compared with traditional services, consumers tend to focus more on their subjective experience when utilizing these services. In the meantime, the problem of power consumption is particularly prominent in 5G and beyond. The traditional design of physical layer (PHY) receiver is based on maximizing spectrum efficiency or minimizing error, but this will no longer be the best after considering energy efficiency and these new-coming services. Therefore, this paper uses quality of experience (QoE) as the optimization criterion of the PHY algorithm. In order to establish the relationship between PHY and QoE, this paper models the end-to-end transmission from UE perspective and proposes a five-layer framework based on hierarchical analysis method, which includes system-level model, bitstream model, packet model, service quality model and experience quality model. Real data in 5G network is used to train the parameters of the involved models for each type of services, respectively. The results show that the PHY algorithms can be simplified in perspective of QoE.
Monotonic Gaussian Process for Physics-Constrained Machine Learning With Materials Science Applications

Abstract Physics-constrained machine learning is emerging as an important topic in the field of machine learning for physics. One of the most significant advantages of incorporating physics constraints into machine learning methods is that the resulting model requires significantly less data to train. By incorporating physical rules into the machine learning formulation itself, the predictions are expected to be physically plausible. Gaussian process (GP) is perhaps one of the most common methods in machine learning for small datasets. In this paper, we investigate the possibility of constraining a GP formulation with monotonicity on three different material datasets, where one experimental and two computational datasets are used. The monotonic GP is compared against the regular GP, where a significant reduction in the posterior variance is observed. The monotonic GP is strictly monotonic in the interpolation regime, but in the extrapolation regime, the monotonic effect starts fading away as one goes beyond the training dataset. Imposing monotonicity on the GP comes at a small accuracy cost, compared to the regular GP. The monotonic GP is perhaps most useful in applications where data are scarce and noisy, and monotonicity is supported by strong physical evidence.
DiverGet: a Search-Based Software Testing approach for Deep Neural Network Quantization assessment

Quantization is one of the most applied Deep Neural Network (DNN) compression strategies, when deploying a trained DNN model on an embedded system or a cell phone. This is owing to its simplicity and adaptability to a wide range of applications and circumstances, as opposed to specific Artificial Intelligence (AI) accelerators and compilers that are often designed only for certain specific hardware (e.g., Google Coral Edge TPU). With the growing demand for quantization, ensuring the reliability of this strategy is becoming a critical challenge. Traditional testing methods, which gather more and more genuine data for better assessment, are often not practical because of the large size of the input space and the high similarity between the original DNN and its quantized counterpart. As a result, advanced assessment strategies have become of paramount importance. In this paper, we present DiverGet, a search-based testing framework for quantization assessment. DiverGet defines a space of metamorphic relations that simulate naturally-occurring distortions on the inputs. Then, it optimally explores these relations to reveal the disagreements among DNNs of different arithmetic precision. We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images. We chose the remote sensing DNNs as they’re being increasingly deployed at the edge (e.g., high-lift drones) in critical domains like climate change research and astronomy. Our results show that DiverGet successfully challenges the robustness of established quantization techniques against naturally-occurring shifted data, and outperforms its most recent concurrent, DiffChaser, with a success rate that is (on average) four times higher.
Inferring Speaking Styles from Multi-modal Conversational Context by Multi-scale Relational Graph Convolutional Networks

To support applications of speech-driven interactive systems in various conversational scenarios, text-to-speech (TTS) synthesis needs to understand the conversational context and determine appropriate speaking styles in its synthesized speeches. These speaking styles are influenced by the dependencies between the multi-modal information in the context at both global scale (i.e. utterance level) and local scale (i.e. word level). However, the dependency modeling and speaking style inference at the local scale are largely missing in state-of-the-art TTS systems, resulting in the synthesis of incorrect or improper speaking styles. In this paper, to learn the dependencies in conversations at both global and local scales and to improve the synthesis of speaking styles, we propose a context modeling method which models the dependencies among the multi-modal information in context with multi-scale relational graph convolutional network (MSRGCN). The learnt multi-modal context information at multiple scales is then utilized to infer the global and local speaking styles of the current utterance for speech synthesis. Experiments demonstrate the effectiveness of the proposed approach, and ablation studies reflect the contributions from modeling multi-modal information and multi-scale dependencies.
Efficient Modeling of Future Context for Image Captioning

Existing approaches to image captioning usually generate the sentence word-by-word from left to right, with the constraint of conditioned on local context including the given image and history generated words. There have been many studies target to make use of global information during decoding, e.g., iterative refinement. However, it is still under-explored how to effectively and efficiently incorporate the future context. To respond to this issue, inspired by that Non-Autoregressive Image Captioning (NAIC) can leverage two-side relation with modified mask operation, we aim to graft this advance to the conventional Autoregressive Image Captioning (AIC) model while maintaining the inference efficiency without extra time cost. Specifically, AIC and NAIC models are first trained combined with shared visual encoders, forcing the visual encoder to contain sufficient and valid future context; then the AIC model is encouraged to capture the causal dynamics of cross-layer interchanging from NAIC model on its unconfident words, which follows a teacher-student paradigm and optimized with the distribution calibration training objective. Empirical evidences demonstrate that our proposed approach clearly surpass the state-of-the-art baselines in both automatic metrics and human evaluations on the MS COCO benchmark. The source code is available at: https://github.com/feizc/Future-Caption.
CryptoAR: scrutinizing the trend and market of cryptocurrency using machine learning approach on time series data

&lt;span&gt;Cryptocurrencies are encrypted digital or virtual money used to avoid counterfeiting and double spending. The scope of this study is to evaluate cryptocurrencies and forecast their price in the context of the currency rate trends. A public survey was conducted to determine which cryptocurrency is the most well-known among Bangladeshi people. According to the survey respondents, Bitcoin is the most famous cryptocurrency among the eight digital currencies. After that, we'll explore the four most well-known cryptocurrencies: Bitcoin, Ethereum, Litecoin, and Tether token. The 'YFinance' python package collects our cryptocurrency dataset, and the relative strength index (RSI) is employed to investigate these cryptocurrencies. Autoregressive (AR), moving average (MA), and autoregressive moving average (ARMA) models are applied to our time-series data from 2015-1-1 to 2021-6-1. Using the 'closing' price and a simple moving average (SMA) graph, bitcoin and tether are identified as oversold or overbought cryptocurrencies. We employ the seasonal decomposed technique into the dataset before implementing the model, and the augmented dickey-fuller test (ADF) indicates too much seasonality in the dataset. The autoregressive (AR) model is the most accurate in predicting the price of Bitcoin, Ethereum, Litecoin, and Tether-token, with 97.21%, 96.04%, 95.8%, and 99.91% accuracy, consecutively.&lt;/span&gt;
Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Clouds

To boost a detector for single-frame 3D object detection, we present a new approach to train it to simulate features and responses following a detector trained on multi-frame point clouds. Our approach needs multi-frame point clouds only when training the single-frame detector, and once trained, it can detect objects with only single-frame point clouds as inputs during the inference. For this purpose, we design a novel Simulated Multi-Frame Single-Stage object Detector (SMF-SSD) framework: multi-view dense object fusion to densify ground-truth objects to generate a multi-frame point cloud; self-attention voxel distillation to facilitate one-to-many knowledge transfer from multi- to single-frame voxels; multi-scale BEV feature distillation to transfer knowledge in low-level spatial and high-level semantic BEV features; and adaptive response distillation to activate single-frame responses of high confidence and accurate localization. Experimental results on the Waymo test set show that our SMF-SSD consistently outperforms all state-of-the-art single-frame 3D object detectors for all object classes of difficulty levels 1 and 2 in terms of both mAP and mAPH.
APPTracker: Improving Tracking Multiple Objects in Low-Frame-Rate Videos

Multi-object tracking (MOT) in the scenario of low-frame-rate videos is a promising solution for deploying MOT methods on edge devices with limited computing, storage, power, and transmitting bandwidth. Tracking with a low frame rate poses particular challenges in the association stage as objects in two successive frames typically exhibit much quicker variations in locations, velocities, appearances, and visibilities than those in normal frame rates. In this paper, we observe severe performance degeneration of many existing association strategies caused by such variations. Though optical-flow-based methods like CenterTrack can handle the large displacement to some extent due to their large receptive field, the temporally local nature makes them fail to give correct displacement estimations of objects whose visibility flip within adjacent frames. To overcome the local nature of optical-flow-based methods, we propose an online tracking method by extending the CenterTrack architecture with a new head, named APP, to recognize unreliable displacement estimations. Then we design a two-stage association policy where displacement estimations or historical motion cues are leveraged in the corresponding stage according to APP predictions. Our method, with little additional computational overhead, shows robustness in preserving identities in low-frame-rate video sequences. Experimental results on public datasets in various low-frame-rate settings demonstrate the advantages of the proposed method.
MimCo: Masked Image Modeling Pre-training with Contrastive Teacher

Recent masked image modeling (MIM) has received much attention in self-supervised learning (SSL), which requires the target model to recover the masked part of the input image. Although MIM-based pre-training methods achieve new state-of-the-art performance when transferred to many downstream tasks, the visualizations show that the learned representations are less separable, especially compared to those based on contrastive learning pre-training. This inspires us to think whether the linear separability of MIM pre-trained representation can be further improved, thereby improving the pre-training performance. Since MIM and contrastive learning tend to utilize different data augmentations and training strategies, combining these two pretext tasks is not trivial. In this work, we propose a novel and flexible pre-training framework, named MimCo, which combines MIM and contrastive learning through two-stage pre-training. Specifically, MimCo takes a pre-trained contrastive learning model as the teacher model and is pre-trained with two types of learning targets: patch-level and image-level reconstruction losses.
HyP <sup>2</sup> Loss: Beyond Hypersphere Metric Space for Multi-label Image Retrieval

Image retrieval has become an increasingly appealing technique with broad multimedia application prospects, where deep hashing serves as the dominant branch towards low storage and efficient retrieval. In this paper, we carried out in-depth investigations on metric learning in deep hashing for establishing a powerful metric space in multi-label scenarios, where the pair loss suffers high computational overhead and converge difficulty, while the proxy loss is theoretically incapable of expressing the profound label dependencies and exhibits conflicts in the constructed hypersphere space. To address the problems, we propose a novel metric learning framework with Hybrid Proxy-Pair Loss (HyP$^2$ Loss) that constructs an expressive metric space with efficient training complexity w.r.t. the whole dataset. The proposed HyP$^2$ Loss focuses on optimizing the hypersphere space by learnable proxies and excavating data-to-data correlations of irrelevant pairs, which integrates sufficient data correspondence of pair-based methods and high-efficiency of proxy-based methods. Extensive experiments on four standard multi-label benchmarks justify the proposed method outperforms the state-of-the-art, is robust among different hash bits and achieves significant performance gains with a faster, more stable convergence speed. Our code is available at https://github.com/JerryXu0129/HyP2-Loss.