Reviewerly/exHarmony huggingface dataset:
An Interactive Evolution Strategy based Deep Convolutional Generative Adversarial Network for 2D Video Game Level Procedural Content Generation

The generation of desirable video game contents has been a challenge of games level design and production. In this research, we propose a game player flow experience driven interactive latent variable evolution strategy incorporated with a Deep Convolutional Generative Adversarial Network (DCGAN) for undertaking game content generation with respect to a 2D Super Mario video game. Since the Generative Adversarial Network (GAN) models tend to capture the high-level style of the input images by learning the latent vectors, they are used to generate game scenarios and context images in this research. However, as GANs employ arbitrary inputs for game image generation without taking specific features into account, they generate game level images in an incoherent manner without the specific playable game level properties, such as a broken pipe in the Mario game level image. In order to overcome such drawbacks, we propose a game player flow experience driven optimised mechanism with human intervention, to guide the game level content generation process so that only plausible and even enjoyable images will be generated as the candidates for the final game design and production.
Innovative Shoe-Integrated System Based on Time-of-Flight Range Sensors for Fall Detection on Various Terrains

This letter presents a shoe-integrated fall detection system that is based on seven time-of-flight (ToF) range sensors to identify falls on different terrains. The ToF range sensors, which measure the real-time distance between the sensors and the ground surface, can be used to detect simultaneously the variation of the gait cycle and the geometry of the terrain. Using the statistical features of the ToF range signals and an error-correcting output codes-based model, 17 experimental activities, executed by nine participants, are classified into three classes—activities of daily life, near falls, and falls. The experimental F1-scores for these three classes are 100, 98.17, and 99.07%, respectively. The arrangement of ToF range sensors is simplified to develop a compact shoe-integrated system architecture for fall detection.
On-Policy Deep Reinforcement Learning Approach to Multi Agent Problems

Reinforcement learning approach has been preferred by researchers and scientists in recent years, especially due to its superior performance in robot studies. While smart systems are becoming widespread in technology that develops and changes day by day, communication problems between these systems and the environment are still among the issues that are being studied with great importance. Reinforcement learning determines what action to take in the next step by rewarding the experiences gained from the environment it is in. The most important difference from other machine learning approaches is that it does not need any preliminary data during the training phase. In this study, a deep reinforcement learning method that regulates the movements of three different robots used in limited areas is presented. The performance of the robots has been tested by training this problem with the Policy Gradient algorithm. With the presented method, it was ensured that the robots learn how to act both in the area they are in and in the collision problems against each other with the deep reinforcement learning method. Robots in this kind of mission do not only have a duty, but also need to be trained in terms of safety. In order to solve this problem, training was carried out by positioning fixed obstacles in the training environment. In this way, it performs its duty without hitting the fixed or mobile obstacles in the closed environment. This study was trained in a simulation environment. It is addressed using a concurrent learning approach without any communication between agents. A multi-agent approach based on policy gradient, temporal difference error and actor-critic methods is used. The performance of robots is reported especially by supporting the reinforcement learning approach with deep learning algorithms. Looking at the simulation results, it is seen that robots trained with the Policy Gradient algorithm are successful in cleaning the whole area. In addition, the rewards obtained by the robots at the end of the training are given in detail in section 4.
A Pitman-Yor Process Self-Aggregated Topic Model for Short Texts of Social Media

In recent years, with the rapid growth of social media, short texts have been very prevalent on the internet.Due to the limited length of each short text, word co-occurrence information in this type of documents is sparse.Conventional topic models based on word co-occurrence are unable to distill coherent topics on short texts.A state-of-the-art strategy is self-aggregated topic models which implicitly aggregate short texts into latent long documents.But these models have two problems.One problem is that the number of long documents should be defined explicitly and the inappropriate number leads to poor performance.Another problem is that latent long documents may bring non-sematic word co-occurrence which brings incoherent topics.In this article, we firstly apply the Chinese restaurant process to automatically generate the number of long documents according to the scale of short texts.Then to exclude non-semantic word cooccurrence, we propose a novel probabilistic model generating latent long documents in a more semantically way.Specifically, our model employs a pitman-yor process to aggregate short texts into long documents.This stochastic process can guarantee that the distribution between short texts and long documents following a power-law distribution which can be found in social media like Twitter.Finally, we compared our method with several state-of-the-art methods on four real short texts corpus.The experiment results show that our model performs superior to other methods with the metrics of topic coherence and text classification.
Yoroi: Updatable Whitebox Cryptography

Whitebox cryptography aims to provide security in the whitebox setting where the adversary has unlimited access to the implementation and its environment. In order to ensure security in the whitebox setting, it should prevent key extraction attacks and code-lifting attacks, in which the adversary steals the original cryptographic implementation instead of the key, and utilizes it as a big key. Although recent published ciphers such as SPACE, SPNbox, and Whiteblock successfully achieve security against the key extraction attacks, they only provide mitigation of codelifting attack by the so-called space hardness and incompressibility properties of the underlying tables as the space-hard/incompressible table might be eventually stolen by continuous leakage. The complete prevention of such attacks may need to periodically update the secret key. However, that entails high costs and might introduce an additional vulnerability into the system due to the necessity for the reencryption of all data by the updated key. In this paper, we introduce a new property, denominated longevity, for whitebox cryptography. This property enhances security against code-lifting attacks with continuous leakage by updating incompressible tables instead of the secret key. We propose a family of new whitebox-secure block ciphers Yoroi that has the longevity property in addition to the space hardness. By updating its implementation periodically, Yoroi provides constant security against code-lifting attacks without key updating. Moreover, the performance of Yoroi is competitive with existing ciphers implementations in the blackbox and whitebox context.
Towards Compile-Time-Reducing Compiler Optimization Selection via Machine Learning

Compilers come with a multitude of optimizations to choose from, and the chosen optimizations significantly affect the performance of the code being optimized. Selecting the optimal set of optimizations to apply and determining the order to run them is non-trivial. All of these optimizations closely interact with each other, and an optimization’s ability to improve the code heavily depends on how the previous optimizations modified it. The current approach to solve this is to use a one-size-fits-all optimization sequence, that is designed to perform reasonably well for any given source code. In other words, they are not designed to optimize depending on the nature of the code, which usually results in sub-optimal performance. In this paper, we present preliminary work tackling the problem from the perspective of compile-time by adapting the optimization sequence to cater to different program types. We start by analyzing how the source code interacts with the passes, as well as how the passes interact with each other. We use this information and propose two potential methods driven by machine learning to run customized optimization sequences on the source code. First, we look at how we can use a neural network to predict and skip passes that do not optimize the code to improve compilation time. Second, we look at how we can use clustering and predictive models to select custom pass pipelines. We believe that our approach has the potential to replace the current one-size-fits-all approach, with better optimization sequences that are tailored to perform better depending on the code. At the same time, it will allow testing the potential pipelines thoroughly, a practical requirement to gain confidence in the correctness of the compiler.
Robust Image-based Visual Servoing for Autonomous Row Crop Following with Wheeled Mobile Robots

In this work, we present a new robust vision-based controller for wheeled mobile robots, equipped with a fixed monocular camera, to perform autonomous navigation in agricultural fields accurately. Here, we consider the existence of uncertainties in the parameters of the robot-camera system and external disturbances caused by high driving velocities, sparse plants, and terrain unevenness. Then, we design a robust image-based visual servoing (rIBVS) approach based on the sliding mode control (SMC) method for robot motion stabilization, even under the presence of such inaccuracies and perturbations. The vision-based controller, based on column and row primitives, is slightly modified to include a robustness term into the original feedback control laws to ensure successful row crop reaching and following tasks. We employ the Lyapunov stability theory to verify the stability and robustness properties of the overall closed-loop system. 3D computer simulations are carried out in the ROS-Gazebo platform, an open-source robotics simulator, using a differential-drive mobile robot (DDMR) in an ad-hoc developed row crop environment to illustrate the effectiveness and feasibility of the proposed control methodology.
Hw/Sw Co-Design technique for 2D fast fourier transform algorithm on Zynq SoC

The Two-Dimensional Fast Fourier Transform (2D-FFT) algorithm is used for the study of many modern systems applied for security and biometrics. The adoption of this algorithm, which is a compute intensive task, is limited due to its hardware design complexity. The first objective of this paper is to underline the effect of the hardware/software co-design (Hw/Sw co-design) for the reduction of the processing time and power consumption. Secondly, we propose an innovative architecture for the 2D-FFT algorithm tested on Zynq Soc, which requires less processing time and memory compared to the traditional algorithm. Three implementations (one software and two Hw/Sw co-designs) of the 2D-FFT algorithm using the Zynq SoC are presented in this paper. The first is based on ARM processor. A speedup of 29x is obtained compared to the original implementation thanks to many optimizations. The second is a Hw/Sw co-design solution of the traditional 2D-FFT algorithm introduced on a hybrid platform combining an ARM Cortex-A9 processor with an FPGA. The third is also a Hw/Sw co-design solution using our optimized 2D-FFT algorithm to reach the real-time contraints for high-resolution images (1920 × 1080). It provides a speedup of 1.13x, 3.31x and 96.21x faster than the Hw/Sw co-design implementation of the traditional RC algorithm, the pure software implementations with and without optimizations, respectively.
PointBA: Towards Backdoor Attacks in 3D Point Cloud

3D deep learning has been increasingly more popular for a variety of tasks including many safety-critical applications. However, recently several works raise the security issues of 3D deep nets. Although most of these works consider adversarial attacks, we identify that backdoor attack is indeed a more serious threat to 3D deep learning systems but remains unexplored. We present the backdoor attacks in 3D with a unified framework that exploits the unique properties of 3D data and networks. In particular, we design two attack approaches: the poison-label attack and the clean-label attack. The first one is straightforward and effective in practice, while the second one is more sophisticated assuming there are certain data inspections. The attack algorithms are mainly motivated and developed by 1) the recent discovery of 3D adversarial samples which demonstrate the vulnerability of 3D deep nets under spatial transformations; 2) the proposed feature disentanglement technique that manipulates the feature of the data through optimization methods and its potential to embed a new task. Extensive experiments show the efficacy of the poison-label attack with over 95% success rate across several 3D datasets and models, and the ability of clean-label attack against data filtering with around 50% success rate. Our proposed backdoor attack in 3D point cloud is expected to perform as a baseline for improving the robustness of 3D deep models.
Constrained Path-Planning Control of Unmanned Surface Vessels via Ant-Colony Optimization

This paper proposes a constrained path-planning scheme for the unmanned surface vessel (USV) via ant-colony optimization. First, a multi-grid map is modeled and established along the kinematics of the USV body suffered by limited energy. A typical ant-colony optimization algorithm is afterwards designed to achieve the energy-saving path planning for narrow channels of water areas. Finally, real lake experiments with HUSTER-12s USV are conducted to verify the effectiveness of the proposed path-planning algorithm.
Enabling On-Device Training of Speech Recognition Models With Federated Dropout

Federated learning can be used to train machine learning models on the edge on local data that never leave devices, providing privacy by default. This presents a challenge pertaining to the communication and computation costs associated with clients’ devices. These costs are strongly correlated with the size of the model being trained, and are significant for state-of-the-art automatic speech recognition models.We propose using federated dropout to reduce the size of client models while training a full-size model server-side. We provide empirical evidence of the effectiveness of federated dropout, and propose a novel approach to vary the dropout rate applied at each layer. Furthermore, we find that federated dropout enables a set of smaller sub-models within the larger model to independently have low word error rates, making it easier to dynamically adjust the size of the model deployed for inference.
Gesture Recognition Based on Improved YOLOv4-tiny Algorithm

With the development of human-computer interaction, gesture recognition is becoming more and more important. At the same time, mobile applications are developing rapidly, it is a development trend to implement human-computer interaction technology on the mobile terminal. An improved YOLOv4-tiny gesture recognition algorithm is proposed. Firstly, on the basis of YOLOv4-tiny network, the Spatial Pyramid Pooling(SPP) module is added to integrate the local and global features of the image to enhance the accurate positioning ability of the network. Secondly, a 1×1 convolution is added after the 3 maximum pooling layers of the original YOLOv4-tiny network and the newly added SPP module, which reduces the network parameters and improves the prediction speed of the network. On this basis, the K-means++ algorithm is used to generate an anchor box suitable for detecting gestures to speed up the network detection of gestures. In the gesture dataset NUS-II, compared with the YOLOv3-tiny algorithm and the YOLOv4-tiny algorithm, the improved algorithm mean Average Precision(mAP) is 100%, Frames Per Second(FPS) is 377, which can detect and recognize gestures quickly and accurately. The improved algorithm of this paper is deployed on the Android mobile terminal to realize the real-time gesture detection and recognition on the mobile terminal, which has great research significance for the development of human-computer interaction.
Object Detection on Thermal Images: Performance of YOLOv4 Trained on Small Datasets

Thermal sensors are underrepresented in the field of Advanced Driver Assistance Systems whereas their capabilities to acquire images independently of weather or daytime can be very helpful to achieve optimal pedestrian and vehicle detection.This underrepresentation is due to the small amount of available public datasets.This lack of training samples and the difficulties of building such datasets are a real hurdle to the development of an object detector dedicated to thermal images.Thanks to YOLOv4 and its detection performance, we show in this paper that finetuning this neural network requires few samples to achieve satisfying performance, outperforming the results of stateoftheart detectors.
An Adaptive Deblurring Vehicle Detection Method for High-Speed Moving Drones: Resistance to Shake

Vehicle detection is an essential part of an intelligent traffic system, which is an important research field in drone application. Because unmanned aerial vehicles (UAVs) are rarely configured with stable camera platforms, aerial images are easily blurred. There is a challenge for detectors to accurately locate vehicles in blurred images in the target detection process. To improve the detection performance of blurred images, an end-to-end adaptive vehicle detection algorithm (DCNet) for drones is proposed in this article. First, the clarity evaluation module is used to determine adaptively whether the input image is a blurred image using improved information entropy. An improved GAN called Drone-GAN is proposed to enhance the vehicle features of blurred images. Extensive experiments were performed, the results of which show that the proposed method can detect both blurred and clear images well in poor environments (complex illumination and occlusion). The detector proposed achieves larger gains compared with SOTA detectors. The proposed method can enhance the vehicle feature details in blurred images effectively and improve the detection accuracy of blurred aerial images, which shows good performance with regard to resistance to shake.
On ADMM in Deep Learning: Convergence and Saturation-Avoidance

In this paper, we develop an alternating direction method of multipliers (ADMM) for deep neural networks training with sigmoid-type activation functions (called \textit{sigmoid-ADMM pair}), mainly motivated by the gradient-free nature of ADMM in avoiding the saturation of sigmoid-type activations and the advantages of deep neural networks with sigmoid-type activations (called deep sigmoid nets) over their rectified linear unit (ReLU) counterparts (called deep ReLU nets) in terms of approximation. In particular, we prove that the approximation capability of deep sigmoid nets is not worse than deep ReLU nets by showing that ReLU activation fucntion can be well approximated by deep sigmoid nets with two hidden layers and finitely many free parameters but not vice-verse. We also establish the global convergence of the proposed ADMM for the nonlinearly constrained formulation of the deep sigmoid nets training to a Karush-Kuhn-Tucker (KKT) point at a rate of order ${\cal O}(1/k)$. Compared with the widely used stochastic gradient descent (SGD) algorithm for the deep ReLU nets training (called ReLU-SGD pair), the proposed sigmoid-ADMM pair is practically stable with respect to the algorithmic hyperparameters including the learning rate, initial schemes and the pro-processing of the input data. Moreover, we find that to approximate and learn simple but important functions the proposed sigmoid-ADMM pair numerically outperforms the ReLU-SGD pair.
A Deep-Learning Based Visual Sensing Concept for a Robust Classification of Document Images under Real-World Hard Conditions

This paper's core objective is to develop and validate a new neurocomputing model to classify document images in particularly demanding hard conditions such as image distortions, image size variance and scale, a huge number of classes, etc. Document classification is a special machine vision task in which document images are categorized according to their likelihood. Document classification is by itself an important topic for the digital office and it has several usages. Additionally, different methods for solving this problem have been presented in various studies; their respectively reached performance is however not yet good enough. This task is very tough and challenging. Thus, a novel, more accurate and precise model is needed. Although the related works do reach acceptable accuracy values for less hard conditions, they generally fully fail in the face of those above-mentioned hard, real-world conditions, including, amongst others, distortions such as noise, blur, low contrast, and shadows. In this paper, a novel deep CNN model is developed, validated and benchmarked with a selection of the most relevant recent document classification models. Additionally, the model's sensitivity was significantly improved by injecting different artifacts during the training process. In the benchmarking, it does clearly outperform all others by at least 4%, thus reaching more than 96% accuracy.
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm

Conventional wisdom in pruning Transformerbased language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit.However, under the trending pretrain-and-finetune paradigm, we postulate a counter-traditional hypothesis, that is: pruning increases the risk of overfitting when performed at the fine-tuning phase.In this paper, we aim to address the overfitting problem and improve pruning performance via progressive knowledge distillation with error-bound properties.We show for the first time that reducing the risk of overfitting can help the effectiveness of pruning under the pretrain-and-finetune paradigm.Ablation studies and experiments on the GLUE benchmark show that our method outperforms the leading competitors across different tasks.