Reviewerly/exHarmony huggingface dataset:
Data augmentation by using gans and image transformation in facial emotion classification

Abstract We propose different approach combinations including GANs and image transformation in facial emotion classification especially for FER-2013 dataset. Our goal is not to reach the highest accuracy for this dataset, but to compare and verify the data augmentation effects for several main GAN methods such as Vanilla GAN, CGAN, DCGAN and BiGAN. We used these GAN models to generate fake images and put them to our train dataset. And the data together was through image transformation to train in the CNN model that we designed to perform facial emotion classification. Base on the result when we used some GANs (BiGAN/DCGAN) and image transformation, the accuracy of the model indeed has been improved. The best combination data augmentation approaches in this paper are using image transformation and DCGAN.
Research on compression pruning methods based on deep learning

Abstract In the past decade or so, deep neural networks have continuously refreshed the best performance in tasks such as computer vision and natural language processing, and have become the most concerned research directions. Although deep neural networks have significant performance, they are difficult to deploy on hardware-constrained embedded and mobile devices due to their huge number of parameters, computational and storage costs. Therefore, in recent years, the lightweight design of network structure has gradually become a cutting-edge and popular direction, improving the operation speed and optimizing the storage space under the premise of maintaining the accuracy of the neural network. In this paper, the results achieved by domestic and foreign scholars in deep learning model compression are sorted and classified, and the methods of network pruning, model quantification, knowledge distillation, and lightweight network design are summarized, and the compression effect of related methods on known public models is summarized. Finally, the possible directions, application directions and development trends of future neural network model lightweight research are prospected.
Spatiotemporal Clustering with Neyman-Scott Processes via Connections to Bayesian Nonparametric Mixture Models

Neyman-Scott processes (NSPs) are point process models that generate clusters of points in time or space. They are natural models for a wide range of phenomena, ranging from neural spike trains to document streams. The clustering property is achieved via a doubly stochastic formulation: first, a set of latent events is drawn from a Poisson process; then, each latent event generates a set of observed data points according to another Poisson process. This construction is similar to Bayesian nonparametric mixture models like the Dirichlet process mixture model (DPMM) in that the number of latent events (i.e. clusters) is a random variable, but the point process formulation makes the NSP especially well suited to modeling spatiotemporal data. While many specialized algorithms have been developed for DPMMs, comparatively fewer works have focused on inference in NSPs. Here, we present novel connections between NSPs and DPMMs, with the key link being a third class of Bayesian mixture models called mixture of finite mixture models (MFMMs). Leveraging this connection, we adapt the standard collapsed Gibbs sampling algorithm for DPMMs to enable scalable Bayesian inference on NSP models. We demonstrate the potential of Neyman-Scott processes on a variety of applications including sequence detection in neural spike trains and event detection in document streams.
Detect small object based on FCOS and adaptive feature fusion

Abstract Object identification has always been a difficult issue in the field of computer vision, where the objective is to identify the location and kind of particular objects in images. Convolutional neural networks have achieved significant advancements in accuracy and speed, and this is mostly due to their quick evolution. However, in practical applications, the detection of small objects is still an unresolved problem. Due to the sparse pixel information of small target, it is relatively difficult to extract small target features effectively. Although many methods have made significant progress, the detection rate and recall rate both have space for improvement under various conditions. This study suggests a small object detection technique that utilizes feature map weight self allocation and single-stage detection model FCOS to further increase the recall and accuracy of small object detection., which is used in aerial photography dataset and airport foreign object dataset respectively. The average detection accuracy and recall of the technique are roughly 10% higher than the baseline, as shown by the quantitative and qualitative trial findings, demonstrating the effectiveness of the adaptive feature fusion approach.
Impact of multi-GPUs and hyperparameters on parallel algorithms based on pytorch distributed

Abstract Neural networks, as powerful models for solving many genuine problems, are suffering from the issue of being too computationally intensive due to the excessive size of the models themselves and the large datasets used for training. This paper’s work focuses on a parallel algorithm, experimented on the classical convolutional neural network LeNet-5, aiming to reduce the time it takes for the neural network to train. This applies a massively parallel GPU structure to take full advantage of its high computing power. Among the approaches to improve the performance of parallel algorithms, many efforts have been made to improve the design of parallel structures, while this work focuses more on the enhancement of parallel algorithms by reasonable settings of neural network hyperparameters. This paper tries to approach (1) reducing the running time by testing parallelism with different numbers of GPUs and (2) further investigating the relationship between parallelism and model accuracy degradation and experimentally deriving a parallel scheme that can reduce the training time while guaranteeing accuracy.
A layer-based random SGD for distributed training of RNNs

Abstract Since frequent data communication leads to increased time consumption in distributed training, compression of communication data is one of the effective ways to speed up the whole training process. Many existing compression methods introduce extra time overhead in data compression and decompression, for instance, Top-K and Random-k. There are also methods that can reduce the extra time overhead but suffer from the shortcoming of insufficient generalization. For example, the layer-based compression LR-SGD (Layer-based Random Stochastic Gradient Descent) cannot be applied to the training of recurrent neural networks. In this paper, an improved method based on LR-SGD is proposed. By being divided into virtual layers for compression, this improved method is able to train various networks efficiently. To validate the proposed method, experiments were conducted using different neural networks on a stimulative cluster. The results demonstrate that the proposed method applies well on different networks with low communication overhead and a nice convergence effect.
Research advanced in image style transfer based on deep learning

Abstract Image style transfer aims to change a picture’s style while maintaining the integrity of the image’s original information. Image style transfer techniques based on deep learning have expanded in variety as a result of the quick advancement of artificial intelligence technology, and have made breakthroughs in both transfer accuracy and visualization effect. Focusing on image style transfer based on depth learning, this paper systematically introduces its latest research progress. Specifically, according to the difference of network structure, this paper introduces four representative style transfer methods based on CNN and GAN, including their design ideas, process based, advantages and disadvantages. In addition, this paper also qualitatively and quantitatively compares the transfer results of different algorithms on different datasets. For GAN based method, AnimeGAN has a good conversion effect in style conversion. For the CNN based method, the combination of perceptual loss and feedforward network using perceptual loss is more prominent. Finally, this paper reviews the development and breakthrough of neural network style, and sorts out and discusses the differences and shortcomings of these technologies as well as some unresolved problems.
Learning additional latent representations using variational autoencoders

Abstract A noticeable trend of machine learning is to deal with data of various modalities. Besides multimodal motivation, learning more from general information without forgetting the prior, or incremental learning, could also benefit unimodal machine learning. Human understanding often starts from a simplistic, generic view of the whole problem and then fills in the details and nuances. Incremental learning could enable imitation of such a learning process to learn more and potentially faster. This paper examines the possibility of learning additional latent variables from known latent variables for variational autoencoders. A method is proposed to facilitate learning additional information based on a modified β -TCVAE loss function that incorporates known general mutual information. A qualitative comparison is conducted on the dSprites dataset to evaluate the effect of this modification and the change of network structures on the learned latent, which hints at the structural tendency for β -TCVAE to learn new information from explicit known latent.
Dynamic gesture recognition based on compressed video for UAV control

Abstract As a multifunctional aircraft with small size, low cost and easy control, UAVs can be used in many fields such as gardening, plant protection, mapping, logistics, military, etc. For a more convenient human-machine interaction mode, the user interacts with the UAVs in the form of dynamic gestures. The current traditional approach is to train convolutional neural networks with images as direct inputs to the system for the purpose of controlling UAVs. However, this approach leaves the temporal representation information between images missing, resulting in poor training results or over-reliance on computing resources. Therefore, this paper proposes an efficient processing strategy, that is, for simple gesture tasks, the optimized frame extraction algorithm is adopted to process picture-based gesture recognition; for complex gesture tasks, compressed video is used as system input to complete video-based gesture recognition. Based on the training results in action recognition datasets, UCF-101 and HMDB-51, the efficiency of compressed video in gesture recognition tasks has been verified, which can be applied to UAV dynamic gesture control.
Image compression and reconstruction based on GUI and Huffman coding

Abstract Huffman coding is an important part of image compression technology, the image compression platform is based on GUI, and Huffman is also widely used. This paper introduces the basic principle of the Huffman algorithm, compares it with arithmetic coding and run length encoding, and expounds on the application of these three algorithms in JPEG compression. The AC algorithm combined block-based, fine texture models and adaptive arithmetic coding in the given an example. The RLE algorithm used automatic threshold, direction judgment, and selective value counts to improve its compression efficiency. JPEG algorithm adopted an adaptive quantization table to reduce the distortion rate of image compression. This paper proves the possibility of a better compression rate and distortion rate of the hybrid compression algorithm by demonstrating the improved example of the basic image compression algorithm. In the future, the improved basic algorithms can be combined based on the original JPEG algorithm, and different algorithms can be integrated on the GUI to face different use environments.
Galaxy morphology classification with densenet

Abstract The classification of galaxies has always been an essential topic in astronomy, which can help to understand how galaxies form and evolve. This paper uses an effective deep-learning architecture, DenseNet-201, to classify galaxy morphology. Because galaxies are only concentrated in the center of the images, we preprocess the data in the way of reframing the images from the size of 256 × 256 × 3 to 224 × 224 × 3 which can eliminate all random noises like any other sub-object. The proposed method, DenseNet, connects all layers to each other. By using the DenseBlock+Transition structure, it realizes feature reuse and reduces the number of features, which could improve computational efficiency. We compare DenseNet-201 with VGG16 and MobileNetV2. VGG16 is very neat and contains multiple Conv-&gt;Conv-&gt;Max_pool structures, and the essential part of MobileNet is depthwise separable convolution. The whole network appears to be stacks of deeply detachable convolution modules. Applying DenseNet-201 to the dataset, Galaxy10 DECals, we achieved 84.1% accuracy of classification, higher than VGG16 of 79% and MobileNetV2 of 78%.
The advance of underdetermined linear regression optimization based on implicit bias

Abstract The gradient descent method has the characteristics of easy realization and simple structure. The traditional gradient descent method has many advantages, especially in solving convex optimization problems. In recent years, some researchers have noticed that the gradient descent algorithm is helpful to solve the problem of underdetermined linear regression optimization. Therefore, in order to explore the specific relationship between gradient descent and under-determined linear regression optimization, this paper focuses on a case with a unique finite root loss function and discusses in detail the working principles of the natural-gradient-descent, mirror-descent, and gradient-descent in this case, and proves the auxiliary effect of gradient descent on under-determined linear regression optimization theoretically. How to use the characteristics of the gradient descent algorithm to design a suitable model, and how do we use the idea of the gradient descent algorithm to find an answer for problem of under determined linear regression optimization in the practical application will be the future research work to continue to advance.
Traffic lights recognition in foggy weather based on YOLOv5

Abstract Autonomous driving technology has gradually matured. The main application is unmanned vehicles, which have begun to test on the road. However, there is a critical technical problem with mitigating interfering factors in efficient traffic light detection and recognition. Due to the size of traffic lights and the complex road conditions, the detection accuracy is not up to satisfactory. This paper proposes a Traffic Light Recognition system based on YOLOv5, which has high speed and accuracy. The system also tests foggy data which gain from image processing. The experimental results show that the method achieved 0.9943 in normal conditions and 0.9937 in foggy conditions. Overall, the efficiency of YOLOv5 meets the practical application requirements.
Robust safety zones for manipulators with uncertain dynamics in collaborative robotics

In this paper, an approach for computing online safety zones for collaborative robotics in a robust way, despite uncertain robot dynamics, is proposed. The strategy implements the speed and separation monitoring paradigm, and considers human and robot enclosed in bounding volumes. The human-robot collaboration is monitored by a supervisory controller that guides the robot to stop along a path-consistent trajectory in case of collision danger between human and robot. The size of the robot safety zone is minimized online according to the stop time of the manipulator, and the uncertain robot dynamics is considered using interval arithmetic to ensure compliance with the joint torques limits even in case of imperfect knowledge of the dynamic model parameters. The results verify the effectiveness of the proposed approach, and evaluate the influence of dynamics variations on human-robot collaboration.
Multiple robot motion planning on A* and RRT* algorithm

Abstract In recent years, multi-robot systems have been widely used in different fields. In order to ensure that each robot in the system can reach the target point and complete the task correctly, the path planning of the multi-robot system is significant. The path planning of the multi-robot system can effectively ensure that the robot selects the appropriate target and completes the task, and during the planning, it can ensure that the task completion time of the total system is the shortest or the total cost is the lowest. Experiments are carried out without considering the kinematics and physical properties of the robot itself, and it is assumed that each robot can accurately know its own position information. In this paper, we propose an A* algorithm for target trajectory planning that integrates multiple trajectories and minimizes running costs by task assignment through a linear programming model. At the same time, by adjusting the planning method of the estimated cost, the error caused by the difference between the nonlinear path and the straight line distance is reduced. After experiments, the effectiveness of the linear programming model and the difference in computation time and running time between A* and RRT* algorithms are demonstrated. What’s more, A* outperforms RRT* in completion time, while RRT* has a shorter computation time.
A novel markov random field segmentation method for SAR images

Abstract The synthetic aperture radar(SAR) images is difficult to be segmented by traditional Hidden Markow Random Field(HMRF)-based segmentation method, a novel HMRF-based segmentation method for SAR images is proposed in this article. The method utilizes more spacial information of SAR image model by involving the intensity differences between pixels in the potential function. An initial segmentation result is obtained by K-means method. Then accurate segmentation is carried out by the novel MRF-based segmentation method. The result showed that this method is better than the traditional HMRF one both in noise filtering and miss classification ratio.
Performance analysis of attention mechanism and teacher forcing ratio based on machine translation

Abstract The Seq2seq model has the characteristics that the encoder is used to analyze the input sequence, and the decoder is used to generate the output sequence, which is very suitable for the characteristics of machine translation tasks, making it achieve advanced performance in the field of machine translation. In addition, the attention mechanism has achieved remarkable results in many kinds of natural language processing tasks. The combination of seq2seq model and attention mechanism is the critical research direction in machine translation. To compare the effects of different attention mechanisms on the performance of machine translation models, this paper selects two classical attention mechanisms, including Synthetic Attention and Muti-head Attention, to test machine translation tasks. In addition, for another critical set of the sequence model, teacher-forcing, this paper also tests the influence of different teacher-forcing thresholds on the model performance. The experimental results show that Muti-head Attention has apparent advantages over Synthetic Attention. In addition, the reasonable choice of teacher-forcing also affects the stability of the model in the real environment.
Teacher Educators’ Attitudes Towards Using Digital Technologies for Learning and Teaching: The Case of Slovenia

This study aimed to identify teacher educators’ attitudes towards using digital technologies in teaching and learning and their self-reported proficiency in using digital tools. It also explored whether and how attitudes towards the use of digital technologies in education predict selfreported proficiency in the use of digital technologies. Sixty-one teacher educators completed a “teachers’ attitudes towards the use of digital technologies in education” questionnaire. The results point to the fact that the teacher educators self-reported being more proficient in using digital technologies that do not demand an advanced level of competence; a lower level of proficiency was reported in using digital technologies that included the co-construction and co-creation of teaching and learning materials.
Performance analysis of robustness of BERT model under attack

Abstract With the aim of testing the robustness of machine learning models, this paper tests the performance of five classification models based on IMDB datasets. Furthermore, in this work, two types of sentence embeddings generated by word2vec and BERT are added with the noise of the normal distribution with different intensities. They are fed into the Support Vector Machine for testing. The experimental results show that the performance of the model slowly decreases as the noise intensity is increased. BERT-based sentiment embedding reduces less than Word2vec-based sentiment embedding. This paper believes that this experimental phenomenon can be used as a basis to support the robustness of BERT representation learning. At the same time, this paper also noted that the previous work experimented with perturbating the BERT model by replacing the words in the original text of the IMDB dataset, causing the BERT model performance to drop sharply. However, the experiment in this paper tested the robustness of the BERT model by embedding tampering, and the experimental results were stable. In this paper, it is assumed that word substitution and embedding interference are possible situations when operating the model. Therefore, for the two different experimental phenomena, this paper plans to design more detailed experiments in the next work to check the robustness of the model.
Performance comparison of representative methods for few-shot speech gender analysis

Abstract As one of the basic tasks of the famous and important speech emotion recognition system, gender recognition based on speech analysis has attracted a lot of research interests in recent year. This paper considers the model performance of different machine learning methods on gender recognition in few-shot learning. In this article, we use three types of methods, namely Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Random forest. Among them, in the experiments of the SVM method, we report the linear SVM and nonlinear SVM models separately. Different experiments were performed with respect to the number of features and number of iterations and the results and possible issues were evaluated.