Reviewerly/exHarmony huggingface dataset:
A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors

Numerous methods and applications have been proposed in human activity recognition (HAR). This paper presents a mini-survey of recent HAR studies and our originally developed benchmark datasets of two types using environmental sensors. For the first dataset, we specifically examine human pose estimation and slight motion recognition related to activities of daily living (ADL). Our proposed method employs OpenPose. It describes feature vectors without effects of objects or scene features, but with a convolutional neural network (CNN) with the VGG-16 backbone, which recognizes behavior patterns after classifying the obtained images into learning and verification subsets. The first dataset comprises time-series panoramic images obtained using a fisheye lens monocular camera with a wide field of view. We attempted to recognize five behavior patterns: eating, reading, operating a smartphone, operating a laptop computer, and sitting. Even when using panoramic images including distortions, results demonstrate the capability of recognizing properties and characteristics of slight motions and pose-based behavioral patterns. The second dataset was obtained using five environmental sensors: a thermopile sensor, a CO2 sensor, and air pressure, humidity, and temperature sensors. Our proposed sensor system obviates the need for constraint; it also preserves each subject’s privacy. Using a long short-term memory (LSTM) network combined with CNN, which is a deep-learning model dealing with time-series features, we recognized eight behavior patterns: eating, operating a laptop computer, operating a smartphone, playing a game, reading, exiting, taking a nap, and sitting. The recognition accuracy for the second dataset was lower than for the first dataset consisting of images, but we demonstrated recognition of behavior patterns from time-series of weak sensor signals. The recognition results for the first dataset, after accuracy evaluation, can be reused for automatically annotated labels applied to the second dataset. Our proposed method actualizes semi-automatic annotation, false recognized category detection, and sensor calibration. Feasibility study results show the new possibility of HAR used for ADL based on unique sensors of two types.
Building a Realistic Virtual Simulator for Unmanned Aerial Vehicle Teleoperation

Unmanned Aerial Vehicles (UAVs) support humans in performing an increasingly varied number of tasks. UAVs need to be remotely operated by a human pilot in many cases. Therefore, pilots require repetitive training to master the UAV movements. Nevertheless, training with an actual UAV involves high costs and risks. Fortunately, simulators are alternatives to face these difficulties. However, existing simulators lack realism, do not present flight information intuitively, and sometimes do not allow natural interaction with the human operator. This work addresses these issues through a framework for building realistic virtual simulators for the human operation of UAVs. First, the UAV is modeled in detail to perform a dynamic simulation in this framework. Then, the information of the above simulation is utilized to manipulate the elements in a virtual 3D operation environment developed in Unity 3D. Therefore, the interaction with the human operator is introduced with a proposed teleoperation algorithm and an input device. Finally, a meta-heuristic optimization procedure provides realism to the simulation. In this procedure, the flight information obtained from an actual UAV is used to optimize the parameters of the teleoperation algorithm. The quadrotor is adopted as the study case to show the proposal’s effectiveness.
ModElec

Integrating electronics with highly custom 3D designs for the physical fabrication of interactive prototypes is traditionally cumbersome and requires numerous iterations of manual assembly and debugging. With the new capabilities of 3D printers, combining electronic design and 3D modeling workflows can lower the barrier for achieving interactive functionality or iterating on the overall design. We present ModElec---an interactive design tool that enables the coordinated expression of electronic and physical design intent by allowing designers to integrate 3D-printable circuits with 3D forms. With ModElec, the user can arrange electronic parts in a 3D body, modify the model design with embedded circuits updated, and preview the auto-generated 3D traces that can be directly printed with a multi-material-based 3D printer. We demonstrate the potential of ModElec with four example applications, from a set of game controls to reconfigurable devices. Further, the tool was reported as easy to use through a preliminary evaluation with eight designers.
Geometric Path Enumeration for Equivalence Verification of Neural Networks

As neural networks (NNs) are increasingly introduced into safety-critical domains, there is a growing need to formally verify NNs before deployment. In this work we focus on the formal verification problem of NN equivalence which aims to prove that two NNs (e.g. an original and a compressed version) show equivalent behavior. Two approaches have been proposed for this problem: Mixed integer linear programming and interval propagation. While the first approach lacks scalability, the latter is only suitable for structurally similar NNs with small weight changes.The contribution of our paper has four parts. First, we show a theoretical result by proving that the epsilon-equivalence problem is coNP-complete. Secondly, we extend Tran et al.’s single NN geometric path enumeration algorithm to a setting with multiple NNs. In a third step, we implement the extended algorithm for equivalence verification and evaluate optimizations necessary for its practical use. Finally, we perform a comparative evaluation showing use-cases where our approach outperforms the previous state of the art, both, for equivalence verification as well as for counter-example finding.
Automated Requirements Formalisation for Agile MDE

Model-driven engineering (MDE) of software systems from precise specifications has become established as an important approach for rigorous software development. However, the use of MDE requires specialised skills and tools, which has limited its adoption.In this paper we describe techniques for automating the derivation of software specifications from requirements statements, in order to reduce the effort required in creating MDE specifications, and hence to improve the usability and agility of MDE. Natural language processing (NLP) and Machine learning (ML) are used to recognise the required data and behaviour elements of systems from textual and graphical documents, and formal specification models of the systems are created. These specifications can then be used as the basis of manual software development, or as the starting point for automated software production using MDE.
End-to-end event factuality prediction using directional labeled graph recurrent network

Event factuality prediction is the task of assessing the degree to which an event mentioned in a sentence has happened. However, existing methods usually stack encoders to make factuality predictions given the gold positions of anchor words. In addition, the frequently used encoders, such as bidirectional LSTMS and graph convolution networks, ignore the directional labeled syntactic information while modeling the context. To fill the gap when facing plain text without identifying event anchor words in advance, we investigate the task of end-to-end EFP in this paper. We present the Directional Labeled Graph Recurrent Network, denoted as DLGRN, to solve Event Anchor Detection and Factuality Induction in a multi-task framework. Specifically, we represent sentences as syntactic information graphs. Then, to incorporate directional labeled information, we design edge-tied weights and edge-aware attention mechanism on top of a graph-based recurrently message passing encoder. We further propose to utilize multi-task learning to jointly model Event Anchor Detection and Factuality Induction by optimizing a mixed-objective learning function. We use four widely used factuality prediction benchmarks (i.e., FactBank, Meantime, UW, and UDS-IH2) to evaluate our framework. Our framework achieves state-of-the-art results in the two subtasks, averagely decreasing 17.12% MAE and raising 5.40% Pearson correlation r against the best baseline. In addition, experimental results show that our framework can capture the overall factuality score distributions, and incorporating directional and labeled syntactic information in EFP achieves better performances than the baselines.
Optimizing Distributed DNN Training Using CPUs and BlueField-2 DPUs

The deep learning (DL) training process consists of multiple phases—data augmentation, training, and validation of the trained model. Traditionally, these phases are executed either on the central processing units or graphics processing units in a serial fashion due to lack of additional computing resources to offload independent phases of DL training. Recently, Mellanox/NVIDIA introduced the BlueField-2 data processing units (DPUs), which combine the advanced capabilities of traditional application-specific-integrated-circuit-based network adapters with an array of ARM processors. In this article, we explore how to take advantage of the additional ARM cores on the BlueField-2 DPUs. We propose and evaluate multiple novel designs to efficiently offload the phases of DL training to the DPUs. Our experimental results show that the proposed designs are able to deliver up to 17.5% improvement in overall DL training time. To the best of our knowledge, this is the first work to explore the use of DPUs to accelerate DL training.
A New Grammar for Creating Convolutional Neural Networks Applied to Medical Image Classification

In the last decade, the adoption of Deep Convolutional Neural Networks (CNNs) has been successfully applied to solve computer vision tasks, such as image classification in the medical field. However, the several architectures proposed in the literature are composed of an increasing number of parameters and complexity. Therefore, finding the optimal trade-off between accuracy and model complexity for a given data set is challenging. To help the search for these suitable configurations, this work proposes using a new Context-Free Grammar associated with a Multi-Objective Grammatical Evolution Algorithm that generates suitable CNNs for a given image classification problem. In this structure, the new grammar maps every possible search space for the creation of networks. Furthermore, the Multi- Objective Grammatical Evolution Algorithm used optimizes this search taking into account two objective functions: accuracy and F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> -score. Our proposal was used in three medical image datasets from MedMNIST: PathMNIST, OCTMNIST, and OrganMNIST_Axial. The results showed that our method generated simpler networks with equal or superior performance from state-of-the-art (more complex) networks and others CNNs also generated by grammatical evolution process.
Segmentation of liver tumors in multiphase computed tomography images using hybrid method

The multiphase scan has enabled an improved detection of liver tumors. However, tumor regions and peripheral tissues are difficult to distinguish and delineate owing to their highly similar image features. Moreover, their characteristics vary significantly in different phases. This is challenging when using segmentation methods that are based on unique training models. Herein, a hybrid framework is proposed for liver tumor segmentation in multiphase images. We first develop a cascade region-based convolutional neural network with refined head to locate the tumors. Meanwhile, phase-sensitive noise filtering is introduced to refine the segmentation conducted by a level-set-based framework. This method is sensitive to the intensity contrast but not to the regions of interest, thereby affording better performance in delineating adjacent tumors. In our experiment, the average precision and recall rates are 76.8% and 84.4%, respectively. The intersection over union, true positive rate, and false positive rate are 72.7%, 76.2%, and 4.75%, respectively.
Quantum Most-Significant Digit-First Addition

In recent years, quantum computers have attracted extensive research interests due to their potential capability of solving problems which are not easily solvable using classical computers. In parallel to the constant research aiming at the physical implementation of quantum processors, there is another branch of research developing quantum algorithms for real-life applications, many of which need to perform arithmetic operations. As one of the most important operations, quantum addition has been adopted in Shor's algorithm, quantum linear algebra algorithms and various quantum machine learning applications. Since precision is always a non-trivial issue to determine during the computation, most-significant digit-first quantum addition can be a fundamental operation for variable precision computing. Therefore, this paper proposes the first quantum adder circuit that is able to compute from the most-significant digits, which demonstrates the advantages over the state-of-the-art quantum adders requiring carry propagation to produce results from least-significant digits. We first present a review of quantum addition circuits design, and then propose a novel method to implement quantum most-significant digit-first adders. Scalability and quantitative comparisons for different quantum full adder, quantum carry-ripple adder and quantum most-significant digit-first adder circuits have been investigated, where all circuits are implemented on IBM Qiskit SDK.
A systematic literature review on support vector machines applied to regression

This article aims to identify the current state of the art of the latest research related to models and algorithms in support vector machines for regression. For that, we use the methodology proposed by Kitchenham and Charter, in order to answer the following research questions: Q1: In which research areas is the support vector machine for regression most used? Q2. What optimization models are used to support vector machine for regression? Q3. What algorithms or optimization methods are used to solve support vector machine for regression? Q4. What nonconvex optimization models use support vector machine for regression? Q5. What optimization algorithms are used for nonconvex models to support vector machine for regression? We obtain valuable information about the questions to construct new models and algorithms in this research area.
Analysis of Volleyball Video Intelligent Description Technology Based on Computer Memory Network and Attention Mechanism

There are some problems in the process of video intelligent description and analysis of volleyball, such as poor effective information extraction rate and poor dynamic tracking effect. Based on this, combined with long-term and short-term memory network and attention mechanism, this paper designs an intelligent description model of volleyball video based on deep learning algorithm and studies how to improve the extraction rate of volleyball video information through intelligent detection hardware and image recognition technology. This paper first introduces the application of image recognition technology and deep learning algorithm in the intelligent description of volleyball video, then designs the volleyball video and image recognition model based on deep learning algorithm according to the requirements of volleyball video intelligent description, and selects three correlation factors related to the impact indicators of volleyball skills. This study selects three characteristic parameters associated with volleyball video analysis indexes, namely, take-off, bounce, and hand movement, combined with image sensing hardware assisted sensor network to realize real-time monitoring of action state in volleyball video analysis system. The experimental results show that, compared with the current mainstream sports video intelligent analysis and image recognition methods with data analysis as the core, the intelligent volleyball sports video intelligent description and image recognition system based on the integration of deep learning algorithm and sensor hardware assistance has the advantages of good detection effect, high data effectiveness, low cost, and high efficiency of volleyball sports video analysis. It can effectively improve the efficiency of volleyball video intelligent description.
Frequency Hopping Signal Modulation Recognition Based on Time-Frequency Analysis

Compared with the fixed frequency signal, the carrier frequency of frequency hopping (FH) signal is controlled by the pseudo-random codes, so it has better concealment and anti-interference. As an important parameter of FH communication, the modulation mode of FH signal can provide powerful support for combat response, such as identification of friend or foe attribute, positioning and jamming guidance, intelligence information extraction, etc. However, there is still a big gap in modulation recognition of FH signals at the domestic and foreign countries. In this paper, a modulation recognition method of FH signal based on time-frequency transform is proposed. The time-frequency images of different modulation types of FH signals are obtained by SPWVD time-frequency transform, and then the time-frequency images are denoised by convolution autoencoder. Finally, the denoised images are sent to convolution neural network for feature extraction and classification recognition. Simulation experiments prove that the proposed method achieves a good classification effect at low signal-to-noise ratios (SNRs), and achieves a recognition rate of 93.67% at -2dB.
On the Imaginary Wings: Text-Assisted Complex-Valued Fusion Network for Fine-Grained Visual Classification

Fine-grained visual classification (FGVC) is challenging due to the interclass similarity and intraclass variation in datasets. In this work, we explore the great merit of complex values in introducing an imaginary part for modeling data uncertainty (e.g., different points on the complex plane can describe the same state) and graph convolutional networks (GCNs) in learning interdependently among classes to simultaneously tackle the above two major challenges. To the end, we propose a novel approach, termed text-assisted complex-valued fusion network (TA-CFN). Specifically, we expand each feature from 1-D real values to 2-D complex value by disassembling feature maps, thereby enabling the extension of traditional deep convolutional neural networks over the complex domain. Then, we fuse the real and imaginary parts of complex features through complex projection and modulus operation. Finally, we build an undirected graph over the object labels with the assistance of a text corpus, and a GCN is learned to map this graph into a set of classifiers. The benefits are in two folds: 1) complex features allow for a richer algebraic structure to better model the large variation within the same category and 2) leveraging the interclass dependencies brought by the GCN to capture key factors of the slight variation among different categories. We conduct extensive experiments to verify that our proposed model can achieve the state-of-the-art performance on two widely used FGVC datasets.
Robust Matrix Factorization via Minimum Weighted Error Entropy Criterion

Learning the intrinsic low-dimensional subspace from high-dimensional data is a key step for many social systems of artificial intelligence. In practical scenarios, the observed data are usually corrupted by many types of noise, which brings a great challenge for social systems to analyze data. As a commonly utilized subspace learning technique, robust low-rank matrix factorization (LRMF) focuses on recovering the underlying subspaces in a noisy environment. However, most of the existing approaches simply assume that the noise contaminating the data is independent identically distributed (i.i.d.), such as Gaussian and Laplacian noises. This assumption, though greatly simplifies the underlying learning problem, may not hold for more complex non-i.i.d. noise widely existed in social systems. In this work, we suggest a robust LRMF approach to deal with various types of noise in a unified manner. Different from traditional algorithms, noise in our framework is modeled using an independent and piecewise identically distributed (i.p.i.d.) source, which employs a collection of distributions, instead of a single one to characterize the statistical behavior of the underlying noise. Assisted by the generic noise model, we then design a robust LRMF algorithm under the information-theoretic learning (ITL) framework through a new minimization criterion. By adopting the half-quadratic optimization paradigm, we further deliver an optimization strategy for our proposed method. Experimental results on both synthetic and real data are provided to demonstrate the superiority of our proposed scheme.
Detail Preserving Low Illumination Image and Video Enhancement Algorithm Based on Dark Channel Prior

In low illumination situations, insufficient light in the monitoring device results in poor visibility of effective information, which cannot meet practical applications. To overcome the above problems, a detail preserving low illumination video image enhancement algorithm based on dark channel prior is proposed in this paper. First, a dark channel refinement method is proposed, which is defined by imposing a structure prior to the initial dark channel to improve the image brightness. Second, an anisotropic guided filter (AnisGF) is used to refine the transmission, which preserves the edges of the image. Finally, a detail enhancement algorithm is proposed to avoid the problem of insufficient detail in the initial enhancement image. To avoid video flicker, the next video frames are enhanced based on the brightness of the first enhanced frame. Qualitative and quantitative analysis shows that the proposed algorithm is superior to the contrast algorithm, in which the proposed algorithm ranks first in average gradient, edge intensity, contrast, and patch-based contrast quality index. It can be effectively applied to the enhancement of surveillance video images and for wider computer vision applications.
Traffic sign recognition based on improved YOLOv4

The current driverless technology has great security risks. The traffic sign recognition technology equipped in the roof sensor is easily affected by light and occlusion, resulting in poor detection effect. To solve this problem, this paper proposes a traffic sign recognition model based on improved yolov4. According to the characteristics of weighted bidirectional feature pyramid network, a cross layer connection is added to the traditional yolov4 network, and the weight of the transferred feature map is adjusted in the process of feature fusion. This method can enhance the feature extraction ability of the network. The experimental results show that the proposed method can detect more types of traffic signs, the mAP of TT100K dataset reaches 87.85%, which is 1.03% higher than the traditional yolov4 algorithm, and the frame processing speed reaches 31.25fps, which meets the requirements of real-time detection.