Reviewerly/exHarmony huggingface dataset:
To Transmit or Predict: An Efficient Industrial Data Transmission Scheme With Deep Learning and Cloud-Edge Collaboration

Many computation-intensive industrial applications need to be run in the cloud, which relies on a lot of sharply varying data transmitted from the industrial field. To save the communication bandwidth and ensure data with required accuracy obtained by the cloud, we design a data transmission architecture based on dual prediction scheme and cloud-edge collaboration and a dual-mode algorithm based on deep learning. With the proposed architecture, a deep learning model is deployed and synchronized on the edge and cloud sides. Further, the proposed algorithm can help the cloud for computation with locally predicted data or real-time data from the edge, depending on whether the predicted data are adequately accurate. A physical validation platform including a sensor, an edge gateway, and a cloud server is built, and drastically changing real vibration data are collected to validate the proposed scheme. The results show that the proposed scheme can reduce 88.66% of data transmission while guaranteeing deviations less than 0.1.
Pairwise Metric Learning With Angular Margin for Tag Recommendation

The users on the Internet have been growing exponentially, and tag recommendation can automatically provide users with a selection of tags of interest to meet their personalized needs. Users can utilize these tags to freely annotate their favorite resources, making them efficient and fast in retrieving related resources. Tensor factorization methods are commonly used in tag recommendation at present. These methods model the user × item × tag interactions, transform the latent feature representations of users, items and tags into low-rank matrices and use inner products for prediction. However, the problem of using inner product is that it does not satisfy the triangle inequality, it ignores the distance relationship among entity pairs and cannot capture the fine-grained preference information. Metric learning in recommendation domains focus on using pairwise loss, which assumes that different categories (such as users, items and tags) have fixed margins. Different categories often have different intra-class variations. With fixed margins, it is often difficult to accurately distinguish between positive and negative samples, thus reducing recommendation performance and limiting the expression ability of the model. In this study, the metric learning method is used to explore the distance relationship among user × item × tag triplet, and the existing metric learning based methods (namely LRML, CML, SML) are applied to the tag recommendation. A pairwise metric learning method with angular margin is proposed, named PMLT. The pairwise distance relationship between user-tag and item-tag is modeled for the information of different entities. And an extra angular margin regularizer is added to the original pairwise loss to control the size of angular margin for user-tag and item-tag respectively. The strength of the constrained angular margin regularizer is controlled to dynamically adjust the distance changes of entity. This method constrains the fixed margin and also the angular margin of user-tag and item-tag. Compared with the traditional metric learning method, this method can capture additional relationship structure and has good recommendation performance. Finally, we conducted extensive experiments on two datasets, LastFm and Movielens, and the experimental results showed that the proposed method PMLT outperform the state-of-the-art baseline in the evaluation metrics Recall@N and NDCG@N, and obtain better prediction quality. We also analyze the influence of different parameters and internal components on the performance of the proposed method, which improves the interpretability of the proposed method.
EHNQ: Subjective and Objective Quality Evaluation of Enhanced Night-Time Images

Vision-based practical applications, such as consumer photography and automated driving systems, greatly rely on enhancing the visibility of images captured in night-time environments. For this reason, various image enhancement algorithms (EHAs) have been proposed. However, little attention has been given to the quality evaluation of enhanced night-time images. In this paper, we conduct the first dedicated exploration of the subjective and objective quality evaluation of enhanced night-time images. First, we build an enhanced night-time image quality (EHNQ) database, which is the largest of its kind so far. It includes 1,500 enhanced images generated from 100 real night-time images using 15 different EHAs. Subsequently, we perform a subjective quality evaluation and obtain subjective quality scores on the EHNQ database. Thereafter, we present an objective blind quality index for enhanced night-time images (BEHN). Enhanced night-time images usually suffer from inappropriate brightness and contrast, deformed structure, and unnatural colorfulness. In BEHN, we capture perceptual features that are highly relevant to these three types of corruptions, and we design an ensemble training strategy to map the extracted features into the quality score. Finally, we conduct extensive experiments on EHNQ and EAQA databases. The experimental and analysis results validate the performance of the proposed BEHN compared with the state-of-the-art approaches. Our EHNQ database is publicly available for download at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/site/xiangtaooo/</uri> .
Analysis of the Limitations of Further Improvement of the Efficiency of VVC-CABAC

Hybrid video compression plays an invaluable role in digital video transmission and storage services and systems. It performs several-hundred-fold reduction in the amount of video data, which makes these systems much more efficient. An important element of hybrid video compression is entropy coding of the data. The state-of-the-art in this field is the newest variant of the Context-based Adaptive Binary Arithmetic Coding (CABAC) entropy compression algorithm which recently became part of the new Versatile Video Coding (VVC) technology. This work is a part of research that is currently underway to further improve the CABAC technique. This paper provides analysis of the potential for further improvement of the CABAC by more accurate calculation of probabilities of data symbols. The CABAC technique calculates those probabilities by the use of idea of the two-parameters hypothesis. For the needs of analysis presented in this paper, an extension of the aforementioned idea was proposed which consists of three- and four-parameters hypothesis. In addition, the paper shows the importance of proper calibration of parameter values of the method on efficiency of data compression. Results of experiments show that for the considered in the paper variants of the algorithm improvement the possible efficiency gain is at levels 0.11% and 0.167%, for the three- and four-parameter hypothesis, respectively.
Multisource Information Fusion Network for Optical Remote Sensing Image Super-Resolution

The super-resolution algorithms based on deep learning can effectively increase optical remote sensing image (ORSI) details for further analysis tasks. Deep unfolding methods have been studied in recent years to bridge the gap between optimization-based and learning-based methods, which can take good advantage of the prior knowledge. However, these unfolding methods usually ignore the utilization of intermediate network features between different iteration stages, thereby limiting the performance of super-resolution results. We propose a multi-source information fusion network (MSFNet) for ORSI super-resolution to address this problem. We mainly consider three strategies to enhance the image super-resolution performance, including feature extraction strategy, information fusion strategy, and the structure of the unfolding network. Firstly, image information of various scales is helpful for mining potential features of images for image super-resolution. Therefore, we introduce multi-scale implicit constraints to the objective function. Secondly, we unfold the optimization process into a neural network by alternating direction method of multipliers (ADMM). The network framework follows the ADMM method's iteration process. This unfolding strategy can effectively utilize the prior information for image reconstruction. Thirdly, we propose a row-column decoupling Transformer module based on this unfolding framework for feature fusion. Specifically, the row Transformer block completes the feature fusion of various scales, and the column Transformer block completes the feature fusion of various channels. The fused features are transmitted to the next iteration stage for feature enhancement. And this fusion strategy enables the network to focus on the global information of the features. We perform experiments on three remote sensing image datasets to fully demonstrate the algorithm's effectiveness. Experiment results show that the proposed algorithm can achieve better image reconstruction performance.
Learning Potential in Subgoal-Based Reward Shaping

Human knowledge can reduce the number of iterations required to learn in reinforcement learning. Though the most common approach uses trajectories, it is difficult to acquire them in certain domains. Subgoals, which are intermediate states, have been studied instead of trajectories. Subgoal-based reward shaping is a reward-shaping framework with a sequence of subgoals based on potential-based reward shaping. The potential function which is a component of potential-based reward shaping has a hyperparameter that controls its output. However, it is not easy to select a hyperparameter because its appropriate value depends on the reward function of an environment, and the reward function is unknown. We propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learned potential</i> that parameterizes a hyperparameter and acquires its potential through learning. A value is an expected accumulated reward if an agent follows its policy after the current state and is strongly related to the reward function. With learned potential, we build an abstract state space with a sequence of subgoals and use the value over the abstract states as the potential to accelerate the value learning. N-step TD learning learns the values over the abstract state. We conducted experiments to evaluate the effectiveness of learned potential, and the results indicate its effectiveness compared with a baseline reinforcement learning algorithm and several reward-shaping algorithms. The results also indicate that the participants’ subgoals are superior to subgoals generated randomly with learned potential. We discuss the appropriate number of subgoals for learned potential, that partially ordered subgoal is helpful for learned potential, that learned potential cannot make learning efficient in step penalized rewards, and that learned potential is superior to the non-learned potential in mixed positive and negative rewards.
Enhanced Federated Learning for Edge Data Security in Intelligent Transportation Systems

Federated learning (FL) provides a promising solution to meet the requirements of data privacy and security in intelligent transportation systems (ITS), which enables edge devices and road side units (RSUs) to collaboratively train learning models without exposing the raw data. However, the deep leakage from gradients (DLG) still leads to the risk of divulging the original data. Meanwhile, the existing gradient protection methods based on secure multi-party computation (SMC) result in huge communication overheads and latency, which are difficult to satisfy the real-time demands of both FL and the diverse services in ITS. Focusing on improving edge data security in ITS, this paper proposes an enhanced federated learning (FL) model (SemBroc-RF) with reinforcement learning, which considers the advantages of both end-to-end homomorphic encryption (HE) and SMC. To reduce communication overheads and strengthen data security in RSUs and end devices simultaneously, a partially encrypted secure multi-party broadcast computation algorithm (SemBroc) is designed, which achieves the time complexity O(n) by constructing the decoding function and sharing the gradients among the local models. To improve the model accuracy by gradients aggregation, a FL algorithm (GreFLa) with reinforcement learning is proposed based on the adaptive assigned weight of the local gradients. Theoretical analysis and detailed simulation results verify that SemBroc-RF can effectively prevent gradient leakage. On the MNIST and CIFAR-10 datasets, compared with the benchmark, the accuracy of SemBroc-RF is increased by 3.63% and 1.35%, and the training round of SemBroc-RF is reduced by 70.8% and 45.6%, respectively.
Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

Laymen are not trained in camera control and how to use a vision mixing desk to switch between different cameras in a video production. Specific training and expert knowledge are required. In the non-professional environment, multi-camera recordings of theater performances or other stage performances are therefore difficult to realize. This can be remedied by a production process with high-resolution cameras from which image sections of long shots or medium long shots are cropped manually or automatically in post-production. One approach to automate the generation of suitable image cuts could be to focus on speaker changes so that the person who is speaking is shown in the generated cut. However, these image cuts can appear static and robotic if they are set too precisely. Therefore, the characteristics and habits of professional vision mixers (persons who operate the vision mixing desk) during the editing process are investigated in more detail in order to incorporate them into an automation process. The characteristic features of five different vision mixers are examined, which were used under almost identical recording conditions for theatrical cuts in TV productions. The cuts are examined with regard to their temporal position in relation to pauses in speech, which take place during speaker changes on stage. It is shown that different professional vision mixers set the cuts individually differently before, in or after the pauses in speech. From the analysis of the image cuts, an approach for a model is developed in which the individual characteristics of a vision mixer can be set. With the help of this model, a more human appearance can be given to otherwise exact and robotic cuts, when automating image cuts.
Sensor-Based Calibration of Camera’s Extrinsic Parameters for Stereophotogrammetry

Stereophotogrammetry is a well-recognized technique for structural health monitoring. Before performing any quantitative stereophotogrammetry measurement, the cameras must be calibrated to obtain the intrinsic and extrinsic parameters of the stereovision system. However, when large-sized structures are to be monitored, the calibration process is challenging and limits the use of stereophotogrammetry. In this article, a sensor-based calibration method for determining the extrinsic parameters of a stereovision system is presented and validated. A multisensor board has been developed that synchronizes inertial measurement units (IMUs) and a laser on a single board computer to measure the spatial orientation and the distance of two paired cameras and compute the extrinsic parameters of the stereovision system. The effectiveness of the sensor-based calibration is evaluated through both analytical studies to quantify the effects of performance degradation caused by the sensors’ noise as well as laboratory tests. Results show that the sensor-based calibration is effective in quantifying displacement with errors below 3% when compared to measurements performed using a stereovision system calibrated with the traditional image-based procedure.
Recurrent Residual Networks Contain Stronger Lottery Tickets

Accurate neural networks can be found just by pruning a randomly initialized overparameterized model, leaving out the need for any weight optimization. The resulting subnetworks are small, sparse, and ternary, making excellent candidates for efficient hardware implementation. However, finding optimal connectivity patterns is an open challenge. Based on the evidence that residual networks may be approximating unrolled shallow recurrent neural networks, we conjecture that they contain better candidate subnetworks at inference time when explicitly transformed into recurrent architectures. This hypothesis is put to the test on image classification tasks, where we find subnetworks within the recurrent models that are more accurate and parameter-efficient than both the ones found within feedforward models and than the full models with learned weights. Furthermore, random recurrent subnetworks are tiny: under a simple compression scheme, ResNet-50 is compressed without a drastic loss in performance to 48.55× less memory size, fitting in under 2 megabytes. Code available at: https://github.com/Lopez-Angel/hidden-fold-networks.
Image Inpainting Based on Structural Constraint and Multi-Scale Feature Fusion

When repairing masked images based on deep learning, there is usually insufficient representation of multi-level information and inadequate utilization of long distance features.To solve the problems, this paper proposes a second-order generative image inpainting model based on Structural Constraints and Multi-scale Feature Fusion (SCMFF).The SCMFF model consists of two parts: edge repair network and image inpainting network.The edge repair network combines the auto-encoder with the Dilated Residual Feature Pyramid Fusion (DRFPF) module, which improves the representation of multi-level semantic information and structural details of images, thus achieves better edge repair.Then, the image inpainting network embeds the Dilated Multi-scale Attention Fusion (DMAF) module in the auto-encoder for texture synthesis with the real edge as the prior condition, and achieves fine-grained inpainting under the edge constraint by aggregating the long-distance features of different dimensions.Finally, the edge repair results are used to replace the real edge, and the two networks are fused and trained to achieve end-to-end repair from the masked image to the complete image.The model is compared with the advanced methods on datasets including Celeba, Facade and Places2.The quantitative results show that the four metrics of LPIPS, MAE, PSNR and SSIM are improved by 0.0124-0.0211,3.787-6.829,2.934dB-5.730dBand 0.034-0.132,respectively.The qualitative results show that the edge distribution in the center of the hole reconstructed by the SCMFF model is more uniform, and the texture synthesis effect is more in line with human visual perception.
StochasticFormer: Stochastic Modeling for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization (WS-TAL) aims to identify the time intervals corresponding to actions of interest in untrimmed videos with video-level weak supervision. For most existing WS-TAL methods, two commonly encountered challenges are under-localization and over-localization, which inevitably bring about severe performance deterioration. To address the issues, this paper proposes a transformer-structured stochastic process modeling framework, namely StochasticFormer, to fully investigate finer-grained interactions among the intermediate predictions to achieve further refined localization. StochasticFormer is built on a standard attention-based pipeline to derive preliminary frame/snippet-level predictions. Then, the pseudo localization module generates variable-length pseudo action instances with the corresponding pseudo labels. Using the pseudo "action instance - action category" pairs as fine-grained pseudo supervision, the stochastic modeler aims to learn the underlying interaction among the intermediate predictions with an encoder-decoder network. The encoder consists of the deterministic and latent path to capture the local and global information, which are subsequently integrated by the decoder to obtain reliable predictions. The framework is optimized with three carefully designed losses, i.e. the video-level classification loss, the frame-level semantic coherence loss, and the ELBO loss. Extensive experiments on two benchmarks, i.e., THUMOS14 and ActivityNet1.2, have shown the efficacy of StochasticFormer compared with the state-of-the-art methods.
A Spatial Attention-Based Behavior Coordination and Scaling Rule of Action Outputs of Mobile Robot in Dynamic Environment

With the arrival of an ultra-smart society, there is a need for mobility support robots that can act as flexibly as humans without trial-and-error learning and re-design of controllers in public spaces where people freely come and go. One of the solutions to this problem is to judge spaces that require suitable attention and make decisions based on only time-series observation information of the surrounding environment. Therefore, we propose a spatial attention-based behavior coordination rule to achieve safer avoidance by increasing the weight of avoidance behavior in advance when attention is paid to a moving obstacle, and a scaling rule of action outputs to adapt more quickly even to obstacles with faster moving speeds. We show the effectiveness of the proposed methods by comparing them with conventional methods through experiments using a real mobile robot.
Defending against code injection attacks using Secure Design Pattern

Several software vulnerabilities emerge during the design phase of a software development process, which can be addressed using secure design patterns. However, using these patterns over web application vulnerabilities is comparatively more tricky for developers than using traditional design patterns. Although several practices exist for addressing software security vulnerabilities, they are sometimes difficult to reuse due to their implementation-specific nature. In this study, we discuss the secure design patterns that are intended to prevent vulnerabilities from being accidentally introduced into code or reduce the effects of flaws. The patterns are created by combining current best security design practices and adding security-specific functionality to the existing design patterns. Hence, this work outlines a convenient mechanism for deciding which secure design patterns to use for addressing online application vulnerabilities. We have demonstrated the applicability of our concept over a prevalent database security threat, namely SQL injection.
What Pakistani Computer Science and Software Engineering Students Think about Software Testing?

Software testing is one of the crucial supporting processes of the software life cycle. Unfortunately for the software industry, the role is stigmatized, partly due to misperception and partly due to treatment of the role. The present study aims to analyse the situation to explore what restricts computer science and software engineering students from taking up a testing career in the software industry. To conduct this study, we surveyed 88 Pakistani students taking computer science or software engineering degrees. The results showed that the present study supports previous work into the unpopularity of testing compared to other software life cycle roles. Furthermore, the findings of our study showed that the role of tester has become a social role, with as many social connotations as technical implications.