Reviewerly/exHarmony huggingface dataset:
Facial Expression Recognition of Animated Characters using Deep Learning

We apply computer vision and deep learning techniques to detect occurrences of a character from animated feature films and subsequently classify its emotions. For children on the autism spectrum, being able to identify facial expressions plays a vital role in the improvement of social responsiveness. Animated Language Learning (ALL), an Ireland based company, use manually labelled images from animated feature films, in interactive software, to help teach speech and language skills to children with autism. Our motivation is to help enable the automated detection of characters and labelling of emotions to improve their workflow. To detect occurrences of a character, our method uses a custom Haar classifier. For this paper, we focus on one character. We manually delete false positives and then manually label the occurrences with five emotion labels. We analyse the inter-annotator agreement and find that this emotion classification task is difficult even for humans. We benchmark two new custom CNNs with a performance analysis against pre-trained state-of-the-art deep CNNs. The models were firstly trained and validated on images from the Toy Story movies 1 to 3, and then tested on Toy Story 4 and some related Pixar short films. As an added step, we also visualise the outputs of key neurons to help understand the best model. The experimental results show that one of our developed models out-performs the best baseline model with an accuracy value of 77.65 %.
Penetration Testing on Web Application Using Insecure Direct Object References (IDOR) Method

Security on a network is a crucial aspect along with the increase in the amount of data exchanged on the internet network. Every company or organization is required to be able to always maintain the confidentiality, integrity, and authentication of data on a web application according to international security standards. This is partly due to the increasing dependence of the community on a web application so that the overall security of the system must always be measured and improved. This paper examines the weaknesses and vulnerabilities of the web application by penetration testing using a method in the form of Insecure Direct Object References (IDOR), with a case study using one URL contained in the application. The test results obtained are the tested URLs then show vulnerabilities to Insecure Direct Object References (IDOR).
Accelerating the Delfs–Galbraith Algorithm with Fast Subfield Root Detection

We give a new algorithm for finding an isogeny from a given supersingular elliptic curve $$E/\mathbb {F}_{p^2}$$ to a subfield elliptic curve $$E'/\mathbb {F}_p$$ , which is the bottleneck step of the Delfs–Galbraith algorithm for the general supersingular isogeny problem. Our core ingredient is a novel method of rapidly determining whether a polynomial $$f \in L[X]$$ has any roots in a subfield $$K \subset L$$ , while avoiding expensive root-finding algorithms. In the special case when $$f=\Upphi _{\ell ,p}(X,j) \in \mathbb {F}_{p^2}[X]$$ , i.e., when f is the $$\ell $$ -th modular polynomial evaluated at a supersingular j-invariant, this provides a means of efficiently determining whether there is an $$\ell $$ -isogeny connecting the corresponding elliptic curve to a subfield curve. Together with the traditional Delfs–Galbraith walk, inspecting many $$\ell $$ -isogenous neighbours in this way allows us to search through a larger proportion of the supersingular set per unit of time. Though the asymptotic $$\tilde{O}(p^{1/2})$$ complexity of our improved algorithm remains unchanged from that of the original Delfs–Galbraith algorithm, our theoretical analysis and practical implementation both show a significant reduction in the runtime of the subfield search. This sheds new light on the concrete hardness of the general supersingular isogeny problem (i.e. the foundational problem underlying isogeny-based cryptography), and has immediate implications on the bit-security of schemes like B-SIDH and SQISign for which Delfs–Galbraith is the best known classical attack.
Ultralight-Weight Three-Prior Convolutional Neural Network for Single Image Super Resolution

The task of image super resolution is crucial in many applications, such as computer vision and medical imaging. Conventionally, the task of image super resolution was carried out by formulating it as a constrained optimization problem and then solving it using suitable numerical techniques. However, after the emergence of deep neural networks, the focus of the researchers in this area has been almost entirely on designing deep convolutional neural network architectures that indeed have provided remarkable performance for the task of image super resolution. Even though unified methods of combining the two approaches has a greater potential of providing a superior performance for the task of image super resolution, with the exception of very few works, not much attention has been paid to develop such a unified method for this task. In this article, we propose a three-prior formulation of the optimization problem for image super resolution and develop an ultralight-weight convolutional neural network for its solution. The effectiveness of the proposed formulation of the optimization problem and ultralight-weight convolution neural network architecture for its solution is demonstrated through extensive experimentations of the proposed scheme on benchmark datasets and comparisons of the results with that of the other state-of-the-art ultralight-weight image super resolution networks.
Design of Four Web Crawlers based on Python

The data on websites is an important source of data for both big data analysis and machine learning. Due to the limitation of data crawling on some websites, the general web crawler will be invalid. To facilitate the crawling of data in websites with different structures, this paper introduces four types of web crawlers. Then, based on some third party libraries developed for Python, the corresponding Python programs are designed respectively for these four web crawlers. This paper provides a technical guide for those researchers who want to construct web crawlers quickly.
TCNet: Co-Salient Object Detection via Parallel Interaction of Transformers and CNNs

The purpose of co-salient object detection (CoSOD) is to detect the salient objects that co-occur in a group of relevant images. CoSOD has been significantly prospered by recent advances in convolutional neural networks (CNNs). However, it shows general limitations in modeling long-range feature dependencies, which is crucial for CoSOD. In the vision transformer, the self-attention mechanism is utilized to capture global dependencies but unfortunately destroy local spatial details, which are also essential for CoSOD. To address the above issues, we propose a dual network structure, called TCNet, which can efficiently excavate both local information and global representations for co-saliency learning via the parallel interaction of Transformers and CNNs. Specifically, it contains three critical components, i.e., the mutual consensus module (MCM), the consensus complementary module (CCM), and the group consistent progressive decoder (GCPD). MCM aims to capture the global consensus from high-level features of these two branches as a guide for the following integration of consensus cues of both branches at each level. Next, CCM is designed to effectively fuse the consensus of local information and global contexts from different levels of the two branches. Finally, GCPD is developed to maintain group feature consistency and predict accurate co-saliency maps. The proposed TCNet is evaluated on five challenging CoSOD benchmark datasets using six widely used metrics, showing that our proposed method is superior to other existing cutting-edge methods for co-salient object detection.
A Motion Representation AI Based Video Conference Solution

Video conference services have been widely used today. Bandwidth cost is huge in HD video conference and thus reducing bandwidth is important for video conference services. In our paper, we proposed a real-time AI based cloud video conference solution on Intel Xeon which enables motion representation AI video reconstruction algorithms such as first order motion model (FOMM) to reduce bandwidth. We also proposed a background enhancement method to improve user experiences. The experimental results showed that our solution can achieve about 80% bandwidth reduction of same video quality compared with H.264. Our demo is available on https://github.com/Amygu1994/AIbasedVideoConference/raw/main/Demo_AI_based_Video_Conference_Solution.mp4.
Sentiment Analysis of Reviews in Kazakh With Transfer Learning Techniques

Heavily pretrained transformer models, such as Bidirectional Encoder Representations from Transformers (BERT) or Generative Pre-trained Transformer (GPT), have successfully demonstrated the superior ability to recognize the right sentiments of texts in English or other dominant languages. However, for low-resource languages such as Kazakh, there are no similar models due to the high computational and memory requirements for their training and the lack of labeled datasets. Under this circumstance, transfer learning can be applied to low-resource language using a pretrained multilingual or related-language model. In this paper, we consider two ways to implement the transfer learning strategy: zero-shot learning and fine-tuning. We design experiments to compare these two methods and report the obtained results. Experiments show that in both cases BERT-based multilingual sentiment analysis model performs better than the BERT-based model for Turkish language, and the performance of these models grows after fine-tuning even with a very small number of samples in Kazakh.
Reactive Motion Planning for Rope Manipulation and Collision Avoidance using Aerial Robots

In this work we address the challenging problem of manipulating a flexible link, like a rope, with an aerial robot. Inspired by spraying tasks in construction and maintenance scenarios, we consider the case in which an autonomous end-effector (e.g., a spray nozzle moved by a robot or a human operator) is connected to a fixed point by a rope (e.g., a hose). To avoid collisions between the rope and the environment while the end-effector moves, we propose the use of an aerial robot as a flying companion to properly manipulate the rope away from collisions. The aerial robot is attached to the rope between the end-effector and the fixed point. Assuming no direct control of the end-effector (e.g., when operated by a human), we design a reactive and fast motion planner for the aerial robot. Grounding on the theory of Forced Geometric Fabrics, we design a motion planner that generates trajectories to drive the aerial robot to follow the end-effector, while manipulating the rope to avoid collisions in cluttered environments. To include the complex behavior of the flexible link, we propose a rope model that estimates its real-time state under forces and position-based interactions, as well as collisions with obstacle surfaces. Finally, we evaluate the system behavior and the motion planner performance in simulations, as well as in real-world experiments on an original spray painting application.
Multi-scale feature learning network with channel self-attention for remote sensing single-image super-resolution

ABSTRACTABSTRACTThe performance of the remote sensing image super-resolution (SR) task has been significantly improved by applying deep learning. However, previously reported methods usually require sufficient synthetic datasets obtained by bicubic downsampling for long-term training, which can get better results when the input of the test phase is still the bicubic downsampling low-resolution (LR) image. However, when the input does not meet this bicubic downsampling situation, the performance of the model will be significantly degraded, and the generated SR image will be blurred. In order to better adapt to remote sensing image SR in different situations, this paper proposes a multi-scale feature learning network with channel self-attention for remote sensing single-image super-resolution (MSFLCSA). The proposed MSFLCSA does not need any extra synthetic paired dataset but one LR input image for training. Further, MSFLCSA uses a well-designed channel self-attention multi-scale feature learning network to fully learn the repetitive multi-scale features of the input image to realize remote sensing image SR tasks. Specifically, the proposed MSFLCSA extracts features of different levels through multi-column convolutions with different receptive fields to better learn the multi-scale features inside remote sensing images. In addition, the channel self-attention module constructs the channel dependence relationship between channels, selectively emphasizes the interdependent channel features, and further improves the learned multi-scale feature representation. Various types of qualitative and quantitative experiments have validated the effectiveness of MSFLCSA. Compared with advanced technologies, MSFLCSA has achieved superior performance.KEYWORDS: Channel self-attentionmulti-scale featureremote sensing imageimage super-resolutionsingle image Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis research was supported by the National Natural Science Foundation of China under Grant No.62072468
A Multimodal Method for Chinese Spelling Correction

Chinese spelling check (CSC) is a task that is both challenging and important, with wide applications in many fields. This paper proposes a multimodal approach to CSC problems, which takes comprehensive consideration of three aspects of knowledge for detecting and correcting Chinese spelling errors, namely, glyph, pinyin and semantic. Our method uses vision transformer, GRU+CNN, and Bert to extract the glyph, pinyin and semantic characteristics respectively, and averages the three embeddings to generate the final multimodal representation, which is fed into a transformer to check and correct possible errors. Our method is trained and tested on standard SIGHAN data sets, and the precision, recall and F1 scores on both character-level and sentence-level show that our method has gained significant improvement over BERT.
MDE-Net: Monocular Distance measurement based on detected license plates

In autonomous driving, it is critical to keep safe distance between cars. To fulfill this task, precise distance estimation is mandatory. The existing distance estimation methods are mainly built upon binocular cameras and laser radars, which are, however, either difficult to calibrate or expensive. To resolve these issues, distance estimation based on monocular cameras is pursued, which is less expensive and easier to calibrate. But monocular cameras can provide only a single image where depth information is lost so that it is more challenging to ensure the distance estimation precision compared with other methods. To address this challenge, this paper proposes a neural network for monocular distance estimation, which estimates the distance of objects within a single image in an end-to-end way. More specifically, we fuse an object detection network and the distance estimation methods based on the keyhole imaging principle to form our Distance Estimation Neural Network, which generates multi-objective distance estimation. Meanwhile, to improve the robustness of the method, we add a license plates detection neural network after the object detection network to detect the shape of the license plate, which can calibrate the depression angle of the sensor camera relative to the car by calculating the perspective transformation matrix of the license plate. To compensate the estimation error of our method in the long distance scenario, we propose a compensation strategy to greatly improve the estimation accuracy.
An Efficient FPGA-based Accelerator for Deep Forest

Deep Forest is a prominent machine learning algorithm known for its high accuracy in forecasting. Compared with deep neural networks, Deep Forest has almost no multiplication operations and has better performance on small datasets. However, due to the deep structure and large forest quantity, it suffers from large amounts of calculation and memory consumption. In this paper, an efficient hardware accelerator is proposed for deep forest models, which is also the first work to implement Deep Forest on FPGA. Firstly, a delicate node computing unit (NCU) is designed to improve inference speed. Secondly, based on NCU, an efficient architecture and an adaptive dataflow are proposed, in order to alleviate the problem of node computing imbalance in the classification process. Moreover, an optimized storage scheme in this design also improves hardware utilization and power efficiency. The proposed design is implemented on an FPGA board, Intel Stratix V, and it is evaluated by two typical datasets, ADULT and Face Mask Detection. The experimental results show that the proposed design can achieve around $40 \times$ speedup compared to that on a 40 cores high performance x86 CPU.
SA-CenterNet: Scale Adaptive CenterNet for X-RAY Luggage Image Detection

Using deep-learing based object detection technology to detect contraband is to locate and classify it in X-RAY luggage images. It can circumvent the problem of missed and false detection caused by traditional manual security inspection. This paper proposes a novel anchor-free detector named Scale Adaptive CenterNet (SA-CenterNet), which models the contraband as its center point at multiple scales. Our method uses keypoint estimation to find center points and regresses to object’s sizes to obtain bounding boxes. SA-CenterNet introduces a scale adaptive module which uses multi-level feature maps to predict occluded contraband with the different sizes and the same center point. In additIon, we propose a feature enhancement module (FEM) to better extract the feature of small and obscure contraband. We use lightweight backbone and group convolution in detection head to reduce the amount of calculation and speed up the inference speed. Experiments demonstrate that our method achieves 95.2%AP <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">50</inf> at 105 FPS on our dataset named UNICOMP, which surpasses the state-of-the-art methods. Our method can robustly detect contraband passing the security inspection machine in real time, saving manpower and material resources.
PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

Reconstructing the 3D pose of a person in metric scale from a single view image is a geometrically ill-posed problem. For example, we can not measure the exact distance of a person to the camera from a single view image without additional scene assumptions (e.g., known height). Existing learning based approaches circumvent this issue by reconstructing the 3D pose up to scale. However, there are many applications such as virtual telepresence, robotics, and augmented reality that require metric scale reconstruction. In this paper, we show that audio signals recorded along with an image, provide complementary information to reconstruct the metric 3D pose of the person. The key insight is that as the audio signals traverse across the 3D space, their interactions with the body provide metric information about the body's pose. Based on this insight, we introduce a time-invariant transfer function called pose kernel-the impulse response of audio signals induced by the body pose. The main properties of the pose kernel are that (1) its envelope highly correlates with 3D pose, (2) the time response corresponds to arrival time, indicating the metric distance to the microphone, and (3) it is invariant to changes in the scene geometry configurations. Therefore, it is readily generalizable to unseen scenes. We design a multistage 3D CNN that fuses audio and visual signals and learns to reconstruct 3D pose in a metric scale. We show that our multi-modal method produces accurate metric reconstruction in realworld scenes, which is not possible with state-of-the-art lifting approaches including parametric mesh regression and depth regression.
Decision Trees

Linear and logistic regressions make predictions about numbers, but we also need algorithms to classify instances of data in a certain class, i.e., to label the instance as belonging to a class. The decision tree is our first approach to solve classification problems. However, decision trees can perform regression too, hence their name classification and regression trees (CART). The random forests that we will encounter in a later chapter are powerful variations of CART.
Biometric Electronic Voting Machine Using Fingerprint Sensor and Arduino

Election is essential and unavoidable part of democratic country and also it is a right of each and every person in the country. Everyone is responsible for secure and valid vote during the election. Government plays a major role during the election to achieve a successful election. Electronic voting machine is currently employed during election also it is possible to do misuse and man power is required to verify the details of the voter. This paper overcomes the cons of the existing method using biometric verification. This system ensures more secure, reliable and less manpower than the existing method.
Semi-Asynchronous Hierarchical Federated Learning Over Mobile Edge Networks

Mobile edge network has been recognized as a promising technology for future wireless communications. However, mobile edge networks usually gathering large amounts of data, which makes it difficult to explore data science efficiently. Currently, federated learning has been proposed as an appealing approach to allow users to cooperatively reap the benefits from trained participants. In this paper, we propose a novel Semi-Asynchronous Hierarchical Federated Learning (SAHFL) framework for mobile edge networks that enables elastic edge to cloud model aggregation from data sensing. We further formulate a joint edge node association and resource allocation problem under the proposed SAHFL framework to prevent personalities of heterogeneous devices and achieve communication-efficiency. To deal with our proposed Mixed integer nonlinear programming (MINLP) problem, we introduce a distributed Alternating Direction Method of Multipliers (ADMM)- Block Coordinate Update (BCU) algorithm. With this algorithm, a tradeoff between training accuracy and transmission latency has been derived. Numerical results demonstrate the advantages of the proposed algorithm in terms of training overhead and model performance.
WhyGen

Deep learning has demonstrated great abilities in various code generation tasks. However, despite the great convenience for some developers, many are concerned that the code generators may recite or closely mimic copyrighted training data without user awareness, leading to legal and ethical concerns. To ease this problem, we introduce a tool, named WhyGen, to explain the generated code by referring to training examples. Specifically, we first introduce a data structure, named inference fingerprint, to represent the decision process of the model when generating a prediction. The fingerprints of all training examples are collected offline and saved to a database. When the model is used at runtime for code generation, the most relevant training examples can be retrieved by querying the fingerprint database. Our experiments have shown that WhyGen is able to precisely notify the users about possible recitations and highly similar imitations with a top-10 accuracy of 81.21%. The demo video can be found at https://youtu.be/EtoQP6850To.