Reviewerly/exHarmony huggingface dataset:
Self-Distillation as Instance-Specific Label Smoothing

It has been recently demonstrated that multi-generational self-distillation can improve generalization. Despite this intriguing observation, reasons for the enhancement remain poorly understood. In this paper, we first demonstrate experimentally that the improved performance of multi-generational self-distillation is in part associated with the increasing diversity in teacher predictions. With this in mind, we offer a new interpretation for teacher-student training as amortized MAP estimation, such that teacher predictions enable instance-specific regularization. Our framework allows us to theoretically relate self-distillation to label smoothing, a commonly used technique that regularizes predictive uncertainty, and suggests the importance of predictive diversity in addition to predictive uncertainty. We present experimental results using multiple datasets and neural network architectures that, overall, demonstrate the utility of predictive diversity. Finally, we propose a novel instance-specific label smoothing technique that promotes predictive diversity without the need for a separately trained teacher model. We provide an empirical evaluation of the proposed method, which, we find, often outperforms classical label smoothing.
Certified variational quantum algorithms for eigenstate preparation

Solutions to many-body problem instances often involve an intractable number of degrees of freedom and admit no known approximations in general form. In practice, representing quantum-mechanical states of a given Hamiltonian using available numerical methods, in particular those based on variational Monte Carlo simulations, become exponentially more challenging with increasing system size. Recently quantum algorithms implemented as variational models have been proposed to accelerate such simulations. The variational ansatz states are characterized by a polynomial number of parameters devised in a way to minimize the expectation value of a given Hamiltonian, which is emulated by local measurements. In this study, we develop a means to certify the termination of variational algorithms. We demonstrate our approach by applying it to three models: the transverse field Ising model, the model of one-dimensional spinless fermions with competing interactions, and the Schwinger model of quantum electrodynamics. By means of comparison, we observe that our approach shows better performance near critical points in these models. We hence take a further step to improve the applicability and to certify the results of variational quantum simulators.
Non-iterative Knowledge Fusion in Deep Convolutional Neural Networks

Incorporation of new knowledge into neural networks with simultaneous preservation of the previous knowledge is known to be a nontrivial problem. This problem becomes even more complex when the new knowledge is contained not in new training examples, but inside the parameters (e.g., connection weights) of another neural network. In this correspondence, we propose and test two methods of combining knowledge contained in separate networks. The first method is based on a summation of weights. The second incorporates new knowledge by modification of weights nonessential for the preservation of previously stored information. We show that with these methods, the knowledge can be transferred non-iteratively from one network to another without requiring additional training sessions. The fused network operates efficiently, performing classification at a level similar to that of an ensemble of networks. The efficiency of the methods is quantified on several publicly available data sets in classification tasks both for shallow and deep feedforward neural networks.
Usability and acceptance of truck dashboards designed by drivers: Two participatory design approaches compared to a user-centered design

Previous literature reported that Participatory Design (PD) results in satisfying products and presents other subjective benefits. PD implies an active involvement of users during all steps of the design process, including during concepts specification (contrary to user-centered design, UCD). This study compared the usability and acceptance of truck dashboards resulting either from a PD or from a UCD design processes. Moreover, design concepts from two PD approaches were compared: a collective versus an individual design session. Results showed that concepts resulting from user-centered design and individual design sessions were perceived as more useable and accepted than the concept resulting from the participatory workshop. These findings question PD benefits and encourage to further explore the different PD approaches in the industrial development of truck dashboards.
Document Structure Extraction Using Prior Based High Resolution Hierarchical Semantic Segmentation

Structure extraction from document images has been a long-standing research topic due to its high impact on a wide range of practical applications. In this paper, we share our findings on employing a hierarchical semantic segmentation network for this task of structure extraction. We propose a prior based deep hierarchical CNN network architecture that enables document structure extraction using very high resolution ( $$1800 \times 1000$$ ) images. We divide the document image into overlapping horizontal strips such that the network segments a strip and uses its prediction mask as prior for predicting the segmentation of the subsequent strip. We perform experiments establishing the effectiveness of our strip based network architecture through ablation methods and comparison with low-resolution variations. Further, to demonstrate our network’s capabilities, we train it on only one type of documents (Forms) and achieve state-of-the-art results over other general document datasets. We introduce our new human-annotated forms dataset and show that our method significantly outperforms different segmentation baselines on this dataset in extracting hierarchical structures. Our method is currently being used in Adobe’s AEM Forms for automated conversion of paper and PDF forms to modern HTML based forms.
Least Squares Surface Reconstruction on Arbitrary Domains

Almost universally in computer vision, when surface derivatives are required, they are computed using only first order accurate finite difference approximations. We propose a new method for computing numerical derivatives based on 2D Savitzky-Golay filters and K-nearest neighbour kernels. The resulting derivative matrices can be used for least squares surface reconstruction over arbitrary (even disconnected) domains in the presence of large noise and allowing for higher order polynomial local surface approximations. They are useful for a range of tasks including normal-from-depth (i.e. surface differentiation), height-from-normals (i.e. surface integration) and shape-from-x. We show how to write both orthographic or perspective height-from-normals as a linear least squares problem using the same formulation and avoiding a nonlinear change of variables in the perspective case. We demonstrate improved performance relative to state-of-the-art across these tasks on both synthetic and real data and make available an open source implementation of our method.
Analysis of risk factors in financial supply chain based on machine learning and IoT technology

The financial supply chain is affected by many factors, so an artificial intelligence model is needed to identify supply chain risk factors. This article combines the actual situation of the financial supply chain, improves the traditional machine learning algorithm, and takes the actual company as an example to build a corresponding risk factor recognition model. From the perspective of optimizing the supply chain financial model, this paper combines the functions of the Internet of Things technology and the characteristics of the supply chain financial inventory pledge financing model to design a new type of inventory pledge financing model. The new model makes up for the defects of the original model through the functions of intelligent identification, visual tracking and cloud computing big data processing of the Internet of Things technology. In addition, this study verifies the performance of the system, uses a large amount of data in Internet finance as an object, and obtains the corresponding results through mathematical statistical analysis. The research results show that the model proposed in this paper has a certain effect on the identification and analysis of financial supply chain risk factors.
Am I Done? Predicting Action Progress in Videos

In this article, we deal with the problem of predicting action progress in videos. We argue that this is an extremely important task, since it can be valuable for a wide range of interaction applications. To this end, we introduce a novel approach, named ProgressNet, capable of predicting when an action takes place in a video, where it is located within the frames, and how far it has progressed during its execution. To provide a general definition of action progress, we ground our work in the linguistics literature, borrowing terms and concepts to understand which actions can be the subject of progress estimation. As a result, we define a categorization of actions and their phases. Motivated by the recent success obtained from the interaction of Convolutional and Recurrent Neural Networks, our model is based on a combination of the Faster R-CNN framework, to make framewise predictions, and LSTM networks, to estimate action progress through time. After introducing two evaluation protocols for the task at hand, we demonstrate the capability of our model to effectively predict action progress on the UCF-101 and J-HMDB datasets.
Multi-Dimensional Routing, Wavelength, and Timeslot Allocation (RWTA) in Quantum Key Distribution Optical Networks (QKD-ON)

Currently, with the continuous advancement of network and communication technology, the amount of data carried by the optical network is very huge. The security of high-speed and large-capacity information in optical networks has attracted more and more attention. Quantum key distribution (QKD) provides information-theoretic security based on the laws of quantum mechanics. Introducing QKD into an optical network can greatly improve the security of the optical network. In order to reduce the cost of deployment on QKD infrastructure, quantum signals in QKD and classical signals in optical networks are multiplexed in the same fiber by wavelength-division manner. Moreover, due to the limited wavelength resources in an optical fiber, time-division technology is adopted to construct different kinds of channels in QKD system for efficient utilization of wavelength resources. Under such situation, how to satisfy the security requirements of service requests and complete the efficient scheduling of multi-dimensional resources, i.e., wavelengths and timeslots, is a challenging problem. This paper addresses this problem by considering multi-dimensional routing, wavelength, and timeslot allocation (RWTA) in short-distance quantum key distribution optical networks (QKD-ON), in which any two nodes can directly establish a quantum channel, and the maximum distance between any two nodes is less than the distance that can carry out point-to-point quantum key distribution process. While accommodating services with security requirements in QKD optical networks, to avoid the wavelength time-slot fragmentation caused by the constraints of wavelength consistency and time-slot continuity, we propose a time-window-based security orchestration strategy as well as relative-loss of time continuous compactness based RWTA strategy. We conducted the simulations under various scenarios, e.g., different key updating periods and different distributions on wavelength resources, etc., and the results show that the proposed strategy can achieve better performance compared with the baselines in terms of key success rate, key-updating delay, and blocking probability.
A type of generalization error induced by initialization in deep neural networks

How initialization and loss function affect the learning of a deep neural network (DNN), specifically its generalization error, is an important problem in practice. In this work, by exploiting the linearity of DNN training dynamics in the NTK regime \citep{jacot2018neural,lee2019wide}, we provide an explicit and quantitative answer to this problem. Focusing on regression problem, we prove that, in the NTK regime, for any loss in a general class of functions, the DNN finds the same \emph{global} minima---the one that is nearest to the initial value in the parameter space, or equivalently, the one that is closest to the initial DNN output in the corresponding reproducing kernel Hilbert space. Using these optimization problems, we quantify the impact of initial output and prove that a random non-zero one increases the generalization error. We further propose an antisymmetrical initialization (ASI) trick that eliminates this type of error and accelerates the training. To understand whether the above results hold in general, we also perform experiments for DNNs in the non-NTK regime, which demonstrate the effectiveness of our theoretical results and the ASI trick in a qualitative sense. Overall, our work serves as a baseline for the further investigation of the impact of initialization and loss function on the generalization of DNNs, which can potentially guide and improve the training of DNNs in practice.
High-Profile VRU Detection on Resource-Constrained Hardware Using YOLOv3/v4 on BDD100K

Vulnerable Road User (VRU) detection is a major application of object detection with the aim of helping reduce accidents in advanced driver-assistance systems and enabling the development of autonomous vehicles. Due to intrinsic complexity present in computer vision and to limitations in processing capacity and bandwidth, this task has not been completely solved nowadays. For these reasons, the well established YOLOv3 net and the new YOLOv4 one are assessed by training them on a huge, recent on-road image dataset (BDD100K), both for VRU and full on-road classes, with a great improvement in terms of detection quality when compared to their MS-COCO-trained generic correspondent models from the authors but with negligible costs in forward pass time. Additionally, some models were retrained when replacing the original Leaky ReLU convolutional activation functions from original YOLO implementation with two cutting-edge activation functions: the self-regularized non-monotonic function (MISH) and its self-gated counterpart (SWISH), with significant improvements with respect to the original activation function detection performance. Additionally, some trials were carried out including recent data augmentation techniques (mosaic and cutmix) and some grid size configurations, with cumulative improvements over the previous results, comprising different performance-throughput trade-offs.
Features of forced quarantine distance learning

The article deals with the study of the concept of distance education and its features. It reveals the investigation of the difference between various forms of distance learning, their positive and negative features as well. A number of rapid changes due to the process of modernization have recently influenced mostly all spheres of human lives, including the sphere of education. In its turn, the format of distance learning has gained more popularity by virtue of being more convenient and thus efficient. However, in Ukraine there is still prejudice about non-standard forms of education; the majority of school students is unaware of the possibility to be educated differently, though the existing distance learning schools offer different options. Unfortunately, the changes in education have also been affected by the outbreak of an epidemic because of COVID-19, which made the countries all around the world, including Ukraine, revise the education system and propose its new formats. Our work aims at differentiation of standard distance learning, which can also be called individual study, from forced quarantine distance learning, singling out their pros and cons and forming a conclusion about which form of education is the most effective. The article includes a poll, the results of which are depicted on diagrams, illustrating opinions of Ukrainian school students regarding this issue.
Generative Adversarial Networks: A Literature Review

The Generative Adversarial Networks, as one of the most creative deep learning models in recent years, has achieved great success in computer vision and natural language processing.It uses the game theory to generate the best sample in generator and discriminator.Recently, many deep learning models have been applied to the security field.Along with the idea of "generative" and "adversarial", researchers are trying to apply Generative Adversarial Networks to the security field.This paper presents the development of Generative Adversarial Networks.We review traditional generation models and typical Generative Adversarial Networks models, analyze the application of their models in natural language processing and computer vision.To emphasize that Generative Adversarial Networks models are feasible to be used in security, we separately review the contributions that their defenses in information security, cyber security and artificial intelligence security.Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning

Matrix factorization is a simple and natural test-bed to investigate the implicit regularization of gradient descent. Gunasekar et al. (2018) conjectured that gradient flow with infinitesimal initialization converges to the solution that minimizes the nuclear norm, but a series of recent papers argued that the language of norm minimization is not sufficient to give a full characterization for the implicit regularization. In this work, we provide theoretical and empirical evidence that for depth-2 matrix factorization, gradient flow with infinitesimal initialization is mathematically equivalent to a simple heuristic rank minimization algorithm, Greedy Low-Rank Learning, under some reasonable assumptions. This generalizes the rank minimization view from previous works to a much broader setting and enables us to construct counter-examples to refute the conjecture from Gunasekar et al. (2018). We also extend the results to the case where depth >= 3, and we show that the benefit of being deeper is that the above convergence has a much weaker dependence over initialization magnitude so that this rank minimization is more likely to take effect for initialization with practical scale.
Understanding the Use of Information Technology as a Supporting Media for Student Learning at SMK Negeri 1 Lubuk Pakam

In the industry 4.0 revolution the current desirable data was faster, and practical to acquire. Especially among the students of the vocational high school did much to learn about theoretical understanding and with the practice of fieldwork. But in the performance of the theoretical learning process, students are no longer capable of critical thinking on a matter or a case given by their teachers. On these occasions, students go obout participating in activities to encourage and enlighten students about how to use good information technology. This service was carried out in a vocational high school (SMK) state 1 to 336 students of al the various vocational. This activity is carried out using the method teacher centered, which is to provide material exposure to sudents regarding the use of the internet. To get an information on concering students understanding in using the internet first post test and after service activities have been done pre-test for. As a result of this activity, which students unutially assume that either the articles or information they obtain over the internet are just as good for job completion needs without sorting things out first, and once this dedicated activity opens up the minds of students that the informations can open up new opportunies for them to complete their education in vocational high school state 1 Lubuk Pakam.
Reconstruction of Hidden Representation for Robust Feature Extraction

This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, which can be viewed as a guiding principle for reconstructing the input. Additionally, when the input is corrupted with noises, the reconstruction error of the corrupted input also can not be lower than a lower bound. 2) The reconstruction of a hidden representation achieving its ideal situation is the necessary condition for the reconstruction of the input to reach the ideal state. 3) Minimizing the Frobenius norm of the Jacobian matrix of the hidden representation has a deficiency and may result in a much worse local optimum value. We believe that minimizing the reconstruction error of the hidden representation is more robust than minimizing the Frobenius norm of the Jacobian matrix of the hidden representation. Based on the above analysis, we propose a new model termed Double Denoising Auto-Encoders (DDAEs), which uses corruption and reconstruction on both the input and the hidden representation. We demonstrate that the proposed model is highly flexible and extensible and has a potentially better capability to learn invariant and robust feature representations. We also show that our model is more robust than Denoising Auto-Encoders (DAEs) for dealing with noises or inessential features. Furthermore, we detail how to train DDAEs with two different pre-training methods by optimizing the objective function in a combined and separate manner, respectively. Comparative experiments illustrate that the proposed model is significantly better for representation learning than the state-of-the-art models.
Visual Servoing of Flying Robot Based on Fuzzy Adaptive Linear Active Disturbance Rejection Control

Visual servoing is the most common control scheme for military and civil robots. For the flying robot, a nonlinear and strongly coupled system, the traditional control methods have the problems of poor adaptive ability and sensitivity to the disturbance in the complex environment. This brief deals with the disadvantages of slow response speed and weak anti-interference ability of flying robot visual servoing by using a fuzzy adaptive linear active disturbance rejection control (FLADRC) scheme. To improve the response speed and robustness of the system, the effect of the controller parameters on the performance is analyzed and quantified into fuzzy rules, and the fuzzy adaptive method is used to adjust the parameters of the linear extended state observer in real-time. Furthermore, a high-speed and high-precision visual positioning algorithm based on YOLOv4 and efficient large-scale stereo matching (ELAS) is proposed. Finally, the performances of FLADRC, LADRC, PID, and Fuzzy-PID are compared by simulation experiments, and the effectiveness of the proposed scheme is proved.