Reviewerly/exHarmony huggingface dataset:
CNN-based End-to-end Autonomous Driving on FPGA Using TVM and VTA

This paper presents a method reducing inference time and maintaining inference accuracy in autonomous driving using TVM and Versatile Tensor Accelerator (VTA) on Field Programmable Gate Array (FPGA). We focus on End-to-end deep neural networks (DNNs) that directly calculate throttle and steering values of cars using camera images to realize autonomous driving. This network is highly accurate in that it does not add any artificial features. However, real-time implementation of autonomous driving DNNs in embedded systems is problematic due to the limited computational resources and electric power. To address this problem, we implemented the network on an FPGA using TVM and VTA. We modified the network using TVM to (1) reduce the number of bits in the neural network parameters from float32 to int8, (2) schedule the matrix computation in hardware, and (3) optimize the operators, tensors, and hardware parameters to maximize the performance of the neural network at runtime. We measured inference time and accuracy of the CPU and CPU + FPGA implementations on the same board. The experiment shows that CPU+FPGA reduced the inference time by 61%, with a 1 % decrease in inference accuracy than CPU implementation. We conclude that FPGA implementation of the end-to-end autonomous driving network can reduce the inference time and maintain the inference accuracy.
Path Planning for an Omnidirectional Mobile Robot Based on Modified A* Algorithm with Energy Model

A path planning framework for an omnidirectional mobile robot based on modified A * algorithm with energy model is proposed. The kinematic model of the omnidirectional mobile robot is established, and the energy consumption model of the omnidirectional mobile robot during motion is established based on the detailed derivation of formulas and used to improve the A* algorithm. The derived results show that the extra power consumed in steering movement is only related to the steering angle.
Privacy-Preserving Neural Network Based on Multi-key NTRU Cryptosystem

Fully Homomorphic Encryption (FHE) is a cryptosystem supporting arbitrary computations over encrypted data. Gentry is one of the foregoers to put forward the concept of full homomorphism who developed the first version of efficient FHE schemes. FHE provides important ideas and directions for the study of privacy-preserving. In my paper, we present a multi-key FHE scheme in virtue of NTRU (Number Theory Research Unit) to implement a neural network, which is capable of preventing the training data from revealing. Compared with the traditional schemes, the neural network of our system works over ciphertexts with semantic security. Meanwhile, the cryptosystem scheme is in the form of multi-keys, thus the data owned by different providers, model trainers, or servers can be independently encrypted but jointly computed. Thereafter, in contrast to federated learning, our scheme does not require any local training. To testify our method, we finally selected a set of commonly used parameters and designed a neural network based on NTRU encryption. And used the physical machine to make encryption predictions for encoding the MNIST dataset, the prediction accuracy of our privacy model is 80%.
Handwriting Recognition Based on Resnet-18

With the development of picture recognition, the number of models of recognizing hand-written images are increasing explosively. In order to find higher-precision and better solutions, more and more people begin to pay attention to improving such models. The complex structure and performance loss of many models limits the growth of Handwriting Recognition. To tackle this issue, we employ Resnet-18 whose structure is optimized and simpler. In the experiment, using Resnet-18 to train this model can not only consider the accuracy, but also ensures that the quantity of parameters is acceptable. With the assist of OpenCV, the numbers can be located by a frame straightforward. The proposed model achieved 99.3% Accuracy when it was tested with 10000 images in the MINIST dataset. Furthermore, this mode can recognize 100 numbers (written by human) almost without making a mistake.
Handwriting Recognition System Leveraging Vibration Signal on Smartphones

The efficiency of human-computer interaction is greatly hindered by the small size of the touch screens on mobile devices, such as smart phones and watches. This has prompted widespread interest in handwriting recognition systems, which can be divided into active and passive systems. Active systems require additional hardware devices to perceive movements of handwriting or the tracking accuracy is not adequate for handwriting recognition. Passive methods use the acoustic signal of pen rubbing and are susceptible to environmental noise (above 60 <inline-formula><tex-math notation="LaTeX">$dB$</tex-math></inline-formula> ). This paper presents a novel handwriting recognition system based on vibration signals detected by the built-in accelerometer of smartphones. The proposed scheme is implemented in three stages: signal segmentation, signal recognition, and word suggestion. <i>VibWriter</i> is highly resistant to interferences since the normal environmental noise (below 70 <inline-formula><tex-math notation="LaTeX">$dB$</tex-math></inline-formula> ) will not cause the vibration of the accelerometer. Extensive experiments demonstrated the efficacy of the system in terms of accuracy in letter recognition (75.3%), word recognition (86.4%) and number recognition (79%) in a variety of writing positions under a variety of environmental conditions.
Thresholding Procedure via Barzilai-Borwein Rules for the Steplength Selection in Stochastic Gradient Methods

A crucial aspect in designing a learning algorithm is the selection of the hyperparameters (parameters that are not trained during the learning process). In particular the effectiveness of the stochastic gradient methods strongly depends on the steplength selection. In recent papers [9, 10], Franchini et al. propose to adopt an adaptive selection rule borrowed from the full-gradient scheme known as Limited Memory Steepest Descent method [8] and appropriately tailored to the stochastic framework. This strategy is based on the computation of the eigenvalues (Ritz-like values) of a suitable matrix obtained from the gradients of the most recent iterations, and it enables to give an estimation of the local Lipschitz constant of the current gradient of the objective function, without introducing line-search techniques. The possible increase of the size of the sub-sample used to compute the stochastic gradient is driven by means of an augmented inner product test approach [3]. The whole procedure makes the tuning of the parameters less expensive than the selection of a fixed steplength, although it remains dependent on the choice of threshold values bounding the variability of the steplength sequences. The contribution of this paper is to exploit a stochastic version of the Barzilai-Borwein formulas [1] to adaptively select the endpoints range for the Ritz-like values. A numerical experimentation for some convex loss functions highlights that the proposed procedure remains stable as well as the tuning of the hyperparameters appears less expensive.
Intelligent Network Security Situation Prediction Method Based on Deep Reinforcement Learning

The prediction of network security situations is the most basic link in the process of intelligent learning, which supports the further development of artificial intelligence technology and the development of new fields. With the help of deep reinforcement learning technology, the prediction effect of network desktops is further improved, which also guarantees the accuracy of the final prediction results. Therefore, an intelligent network security situation prediction method based on deep reinforcement learning is proposed. Calculate the network interference efficiency factor, create the differentiated time series prediction structure, construct the normalized deep reinforcement learning prediction model, expand the security situation prediction evolution under deep reinforcement learning, and use the hybrid optimization method to predict the network security situation. The experimental results show that compared with the traditional time-domain analysis test group, the prediction error of the deep reinforcement learning prediction group is relatively small, indicating that the prediction effect is better and has potential application significance.
The Reiterated Neural Network Parametric Identification of Nonlinear Dynamic Models of Objects

We posed a problem and developed an algorithm for the neural network parametric identification of nonlinear dynamic models of objects with a computational experiment, the formation of training samples on its basis and the subsequent sequential training of two neural networks. Neural networks perform bijective mapping of object parameters into the original model to the output variables of the second neural network. Sequential training or sequential bijective identification of a neural network consists in preliminary training of the first neural network based on experimental data and using its synaptic coefficients to train the second neural network. An example of parametric identification of a nonlinear dynamic model will be a 600 W high pressure sodium lamp. The computational experiment was carried out in the MATLAB environment. The computational experiment technique and the results of experimental studies are presented in the article. Taking into account the good approximating ability of neural networks, the proposed algorithm can be considered as an effective method for parametric identification of nonlinear models.
Supervised Contrastive Learning for Facial Kinship Recognition

Vision-based kinship recognition aims to determine whether the face images have a kin relation. Compared to traditional solutions, the vision-based kinship recognition methods have the advantages of lower cost and being easy to implement. Therefore, such technique can be widely employed in lots of scenarios including missing children search and automatic management of family album. The Recognizing Families in the Wild (RFIW) Data Challenge provides a platform for evaluation of different kinship recognition approaches with ranked results. We propose a supervised contrastive learning approach to address three different kinship recognition tracks (i.e., kinship verification, tri-subject verification, and large-scale search-and-retrieval) announced in the RFIW 2021 with the 2021 FG. Our results on three tracks of 2021 RFIW challenge achieve the highest ranking, which demonstrate the superiority of the proposed solution.
Privacy preserving and secure machine learning

Privacy in Machine Learning is a fundamentally important issue that practitioners must keep in mind while developing models. This paper presents the various methods that can be used to defend models against attack that undermine privacy and safety of the data utilized to generate models. These methods help prevent attacks against both the trained model as well as the underlying training data used by the model. The solutions this paper explores include differential privacy and homomorphic encryption which defend the training data while it is being used to train the model while machine unlearning empowers the data scientist to remove training samples post training.
CMF Net: Detecting Objects in Infrared Traffic Image with Combination of Multiscale Features

Infrared image target detection has always been a hot topic of research, but there is still little research on infrared image target detection in the field of transportation. In this paper, we use the idea of transfer learning to transfer the target detection framework in the visible domain of deep learning to the infrared domain, and propose the target detection model CMF Net based on multi-scale feature fusion. CMF Net uses two multi-scale feature extraction mechanisms and features fusion, so that the final output feature map of the backbone network contains not only low-level visual features which are beneficial to target localization, but also high-level semantic features which are beneficial to target recognition, and can adapt to multi-scale features of the target. The experiment verified the advantages of CMF Net, and its mAP on the test data of the infrared image data set FLIR reached about 71%. This result is an increase of about 13% compared to Faster R-CNN, an increase of about 6% compared to YOLO3, and an increase of about 17% compared to SSD.
FPGA Implementation of a Interval Type-2 Fuzzy Inference for Quadrotor Attitude Control

In this paper, we propose an FPGA implementation of Interval Type-2 Fuzzy Inference (IT2FI) for attitude control of quadrotors. Since IT2FI is suitable for parallel processing, the processing time can be significantly reduced even on low-power FPGA devices. In the proposed system, the IT2FI module is applied to each of the roll, pitch and yaw axis, and the operation amount of the quadrotor motor is obtained from each output value.
Hearing Faces: Target Speaker Text-to-Speech Synthesis from a Face

The existence of a learnable cross-modal association between a person's face and their voice is recently becoming more and more evident. This provides the basis for the task of target speaker text-to-speech (TTS) synthesis from face ref-erence. In this paper, we approach this task by proposing a cross-modal model architecture combining existing unimodal models. We use Tacotron 2 multi-speaker TTS with auditory speaker embeddings based on Global Style Tokens. We trans-fer learn a FaceNet face encoder to predict these embeddings from a static face image reference instead of a voice reference and thus predict a speaker's voice and speaking characteristics from their face. Compared to Face2Speech, the only existing work on this task, we use a more modular architecture that allows the use of openly available and pretrained model components. This approach enables high-quality speech synthesis and allows for an easily extensible model architecture. Ex-perimental results show good matching ability while retaining better voice naturalness than Face2Speech. We examine the limitations of our model and discuss multiple possible av-enues of improvement for future work.
Latent Factor Modeling of Perceived Quality for Stereoscopic 3D Video Recommendation

Numerous stereoscopic 3D movies are released every single year to movie theaters and they evidently generate large revenues. Despite the notable improvements in stereo capturing and 3D video post-production technologies, stereoscopic artefacts continue to appear even in high-budget films. Existing automatic 3D video quality measurement tools can detect distortions in stereoscopic images and videos, but they fail to determine the viewer’s subjective perception of those arte-facts, and how these distortions affect their choices and the overall visual experience. In this paper, we introduce a novel recommendation system for stereoscopic 3D movies based on a latent factor model that meticulously analyzes the viewer’s subjective ratings and the influence of 3D video distortions on their personal preferences. To the best knowledge of the authors, this is definitely a first-of-its-kind model that recommends 3D movies based on quality ratings. It takes the correlation between the viewer’s visual discomfort and the perception of stereoscopic artefacts into account. The proposed model is trained and tested on the benchmark Nama3ds1-cospad1 and LFOVIAS3DPh2 S3D video quality assessment datasets. The experiments highlight the practical efficiency and considerable performance of the resulting matrix-factorization-based recommendation system.
A Study to Improve Satisfaction in Blended Learning - Focused on New Methods of S-University for Quality Control in the Era of COVID-19

COVID-19 has forced tremendous changes in university education. Since the outbreak of COVID-19, the proportion of online education has been strengthened. In the process of revolution, the interest of university education officials is focused on finding ways to improve the satisfaction of Blended Learning. The work performance of those involved in educational innovation at S-University located in the metropolitan area in terms of quality management became the basis of this study. Based on the expanded university autonomy after COVID-19, the results of the new method were confirmed, and the results were statistically verified. Previous studies have emphasized interaction as an important factor influencing the success or failure of online learning. First of all, it was confirmed that student digital learning competency and student-faculty interaction were also related to class satisfaction in S University. Then, while controlling these variables, the correlation of new methods affecting class satisfaction was explored. The first method was to allow each student to learn on their own by using lecture videos about subjects they did not take. The second method was to produce a class summary video summarizing the face-to-face class and provide it to the students. As a result of the study, it was confirmed that both methods had a significant relationship with respect to class satisfaction in all categories of subject types. However, according to the three class types, the relative importance of the new class management method for class satisfaction was not consistent. In the era of Blended Education, follow-up studies to find another variable that can improve higher education class satisfaction should be continued.
Private Multi-Group Aggregation

We study the differentially private multi-group aggregation (PMGA) problem. This setting involves a single server and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> users. Each user belongs to one of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> distinct groups and holds a discrete value. The goal is to design schemes that allow the server to find the aggregate (sum) of the values in each group (with high accuracy) under communication and local differential privacy constraints. The privacy constraint guarantees that the user’s group remains private. This is motivated by applications where a user’s group can reveal sensitive information, such as his religious and political beliefs, health condition, or race. We propose a novel scheme, dubbed Query and Aggregate (Q&A) for PMGA. The novelty of Q&A is that it is an interactive aggregation scheme. In Q&A, each user is assigned a random query matrix, to which he sends the server an answer based on his group and value. We characterize the Q&A scheme’s performance in terms of accuracy (MSE), privacy, and communication. We compare Q&A to the Randomized Group (RG) scheme, which is non-interactive and adapts existing randomized response schemes to the PMGA setting. We observe that typically Q&A outperforms RG, in terms of privacy vs. utility, in the high privacy regime.
Weak Zero-Knowledge beyond the Black-Box Barrier

The round complexity of zero-knowledge protocols is a long-standing open question and is yet to be settled under standard assumptions. So far, the question has appeared equally challenging for relaxations such as weak zero-knowledge and witness hiding. Like full-fledged zero- knowledge, protocols satisfying these relaxed notions under standard assumptions have at least four messages. The difficulty in improving round complexity stems from a fundamental barrier: none of these notions can be achieved in three messages via reductions (or simulators) that treat the verifier as a black box. We introduce a new non-black-box technique and use it to obtain the first protocols that cross this barrier under standard assumptions. Our main results are (1) weak zero-knowledge for NP in two messages, assuming quasi-polynomially secure fully homomorphic encryption and other standard primitives (known from quasi-polynomial hardness of learning with errors) as well as subexponentially secure one-way functions; and (2) weak zero-knowledge for NP in three messages under standard polynomial assumptions (following, for example, from fully homomorphic encryption and factoring). We also give, under polynomial assumptions, a two-message witness-hiding protocol for any language L in NP that has a witness encryption scheme. This protocol is also publicly verifiable. Our technique is based on a new homomorphic trapdoor paradigm, which can be seen as a non-black-box analogue of the classic Feige--Lapidot--Shamir trapdoor paradigm.
Domain Knowledge aided Neural Network for Wireless Channel Estimation

Channel estimation for Orthogonal Frequency Division Multiplexing (OFDM) transmission is well investigated with model based approaches. Recent effort also explores the data driven approaches to exploit the capabilities of Neural Networks (NNs) to estimate the channel. These models are mostly being developed as black box without any anchor to the theory of wireless signal propagation. We propose a NN model, where the structure and parameters are derived from the domain knowledge of wireless signal and channel characteristics. Our model is developed in two stages: the first stage handles the noise reduction, while the second stage extracts the channel characteristics to reduce error caused by multipath. We have used the knowledge of signal to noise ratio and subcarrier correlation due to channel delay spread to empower the two stages of the proposed model. Our results also show that induction of domain knowledge results in reduction of data dependency by 60&#x0025;. Our model outperforms the practical model based methods as well as blind data driven approaches. It achieves &#x223C;10 dB improvement over Least Square channel estimation.
Identifying latent components of the TINAR(1) model

In this paper we give a solution for the problem of identifying and predicting latent components of the integer-valued time series with skewed Skellam marginal distribution. At the beginning, expressions for latent components identification and prediction are derived. These expressions give us a possibility of revealing the values of two hidden, immeasurable components which affect the integer-valued time series with skewed Skellam marginals. Yule-Walker estimators of the unknown parameters are obtained. Also, quality of identification and lag-one prediction is tested on simulated data. At the very end, the model validation is performed in application on the real-life data.
Index and Sort Algorithm Based on FPGA for Sorting Data

Sorting data is one of the most significant problems that have been debated in recent years in order to find the best algorithm for reducing the required time to perform the arrangement. As a result, a hardware accelerator is used to accelerate data processing. In this work, we propose the Index and Sort algorithm (IaSA) as a new sorting algorithm. Our IaSA Hardware architecture is implemented, synthesized, and simulated using Verilog HDL using FPGA vertex-5 series to measure the performance and scalability. The results of our implementations show that IaSA has the best execution time over other existing sorting algorithms. Our proposed algorithm shows execution time smaller than the fastest literature algorithm, odd even merge sort, by about 11.4%, 40.9%, 62.8%, and 41.1% for 4, 8, 16, and 32 dataset sizes respectively.