Reviewerly/exHarmony huggingface dataset:
Video Interpolation by Event-Driven Anisotropic Adjustment of Optical Flow

Video frame interpolation is a challenging task due to the ever-changing real-world scene. Previous methods often calculate the bi-directional optical flows and then predict the intermediate optical flows under the linear motion assumptions, leading to isotropic intermediate flow generation. Follow-up research obtained anisotropic adjustment through estimated higher-order motion information with extra frames. Based on the motion assumptions, their methods are hard to model the complicated motion in real scenes. In this paper, we propose an end-to-end training method A $$^2$$ OF for video frame interpolation with event-driven Anisotropic Adjustment of Optical Flows. Specifically, we use events to generate optical flow distribution masks for the intermediate optical flow, which can model the complicated motion between two frames. Our proposed method outperforms the previous methods in video frame interpolation, taking supervised event-based video interpolation to a higher stage.
Ransomware and Costa Rica’s national emergency: A defense framework and teaching case

This teaching case describes the 2022 Costa Rican ransomware attack national emergency—presenting stakeholder perspectives of perpetrator and prey along with a proactive cybersecurity defense framework (IMSBI), crisscrossing cryptocurrency laundering and ethics.
BARVINN

We present a DNN accelerator that allows inference at arbitrary precision with dedicated processing elements that are configurable at the bit level. Our DNN accelerator has 8 Processing Elements controlled by a RISC-V controller with a combined 8.2 TMACs of computational power when implemented with the recent Alveo U250 FPGA platform. We develop a code generator tool that ingests CNN models in ONNX format and generates an executable command stream for the RISC-V controller. We demonstrate the scalable throughput of our accelerator by running different DNN kernels and models when different quantization levels are selected. Compared to other low precision accelerators, our accelerator provides run time programmability without hardware reconfiguration and can accelerate DNNs with multiple quantization levels, regardless of the target FPGA size. BARVINN is an open source project and it is available at https://github.com/hossein1387/BARVINN.
Crowd-Powered Photo Enhancement Featuring an Active Learning Based Local Filter

In this study, we address local photo enhancement to improve the aesthetic quality of an input image by applying different effects to different regions. Existing photo enhancement methods are either not content-aware or not local; therefore, we propose a crowd-powered local enhancement method for content-aware local enhancement, which is achieved by asking crowd workers to locally optimize parameters for image editing functions. To make it easier to locally optimize the parameters, we propose an active learning based local filter. The parameters need to be determined at only a few key pixels selected by an active learning method, and the parameters at the other pixels are automatically predicted using a regression model. The parameters at the selected key pixels are independently optimized, breaking down the optimization problem into a sequence of single-slider adjustments. Our experiments show that the proposed filter outperforms existing filters, and our enhanced results are more visually pleasing than the results by the existing enhancement methods. Our source code and results are available at https://github.com/satoshi-kosugi/crowd-powered.
Continuous Control of Autonomous Vehicles using Plan-assisted Deep Reinforcement Learning

End-to-end deep reinforcement learning (DRL) is emerging as a promising paradigm for autonomous driving. Although DRL provides an elegant framework to accomplish final goals without extensive manual engineering, capturing plans and behavior using deep neural networks is still an unsolved issue. End-to-end architectures, as a result, are currently limited to simple driving scenarios, often performing sub-optimally when rare, unique conditions are encountered. We propose a novel plan-assisted deep reinforcement learning framework that, along with the typical state-space, leverages a “trajectory-space” to learn optimal control. While the trajectory-space, generated by an external planner, intrinsically captures the agent’s high-level plans, world models are used to understand the dynamics of the environment for learning behavior in latent space. An actor-critic network, trained in imagination, uses these latent features to predict policy and state-value function. Based primarily on DreamerV2 and Racing Dreamer, the proposed model is first trained in a simulator and eventually tested on the FITENTH race car. We evaluate our model for best lap times against parameter-tuned and learning-based controllers on unseen race tracks and demonstrate that it generalizes to complex scenarios where other approaches perform sub-optimally. Furthermore, we show the model’s enhanced stability as a trajectory tracker and establish the improvement in interpretability achieved by the proposed framework.
Social and Emotional Aspect of the Secondary Language Personality Development in Teaching a Professional Foreign Language

Problem and Aim. The article discusses the features of the manifestation of the relationship between the development of various aspects of a linguistic personality and its socio-emotional competencies. The main attention is paid to the socialization of a linguistic personality and its enrichment through the formation of a secondary linguistic personality in the process of learning a foreign language. The purpose of this study is to analyze the factors that influence the successful development of a secondary linguistic personality in the process of its socialization in the conditions of professionally oriented foreign language communication. Methodology. The methodological basis of the study is a complex of general scientific (analysis, synthesis, deduction, induction) and methods of cognition, which made it possible to analyze and synthesize theoretical and practical material. The authors come to the conclusion that students’ social-emotional skills influence students’ interpretation of their experience of foreign language communication, determine positive socialization, effective development of a secondary language personality and create a synergistic effect of students’ stable self-identity.
Image Registration Algorithm for Remote Sensing Images Based on Pixel Location Information

Registration between remote sensing images has been a research focus in the field of remote sensing image processing. Most of the existing image registration algorithms applied to feature point matching are derived from image feature extraction methods, such as scale-invariant feature transform (SIFT), speed-up robust features (SURF) and Siamese neural network. Such methods encounter difficulties in achieving accurate image registration where there is a large bias in the image features or no significant feature points. Aiming to solve this problem, this paper proposes an algorithm for multi-source image registration based on geographical location information (GLI). By calculating the geographic location information that corresponds to the pixel in the image, the ideal projected pixel position of the corresponding image is obtained using spatial coordinate transformation. Additionally, the corresponding relationship between the two images is calculated by combining multiple sets of registration points. The simulation experiment illustrates that, under selected common simulation parameters, the average value of the relative registration-point error between the two images is 12.64 pixels, and the registration accuracy of the corresponding ground registration point is higher than 6.5 m. In the registration experiment involving remote sensing images from different sources, the average registration pixel error of this algorithm is 20.92 pixels, and the registration error of the image center is 21.24 pixels. In comparison, the image center registration error given by the convolutional neural network (CNN) is 142.35 pixels after the registration error is manually eliminated. For the registration of homologous and featureless remote sensing images, the SIFT algorithm can only offer one set of registration points for the correct region, and the neural network cannot achieve accurate registration results. The registration accuracy of the presented algorithm is 7.2 pixels, corresponding to a ground registration accuracy of 4.32 m and achieving more accurate registration between featureless images.
Local Adaptive Prior-Based Image Restoration Method for Space Diffraction Imaging Systems

Thin-film diffractive optical elements (DOEs) have considerable potential to be used in the field of high-resolution remote sensing imaging satellites because of advantages such as a large aperture, small volume, lightness, wide tolerance range of surface shape, and easy replication. However, there are problems associated with thin-film diffraction imaging, including space variation, serious blur, and low contrast, which result in insufficient imaging quality with regard to traditional optical system requirements. To address this, a local adaptive prior-based image restoration method is proposed for thin-film diffraction imaging systems. An entire degraded image was divided into several isohalo regions based on imaging characteristics. Then, the regularization constraints were adaptively selected and updated according to the local scene prior characteristics. Additionally, the system parameters in the corresponding field of view were used as input to restore each subregion. In particular, the diffraction efficiency (DIE) was introduced into the model to remove the nondesign level background radiation. The experimental results show that the proposed algorithm can effectively improve the image quality of a thin-film diffraction imaging system, including space variation correction, clarity enhancement, and background radiation suppression. Furthermore, a DIE of less than 60% was found to significantly impact the final image products.
Towards a Safe Human–Robot Collaboration Using Information on Human Worker Activity

Most industrial workplaces involving robots and other apparatus operate behind the fences to remove defects, hazards, or casualties. Recent advancements in machine learning can enable robots to co-operate with human co-workers while retaining safety, flexibility, and robustness. This article focuses on the computation model, which provides a collaborative environment through intuitive and adaptive human–robot interaction (HRI). In essence, one layer of the model can be expressed as a set of useful information utilized by an intelligent agent. Within this construction, a vision-sensing modality can be broken down into multiple layers. The authors propose a human-skeleton-based trainable model for the recognition of spatiotemporal human worker activity using LSTM networks, which can achieve a training accuracy of 91.365%, based on the InHARD dataset. Together with the training results, results related to aspects of the simulation environment and future improvements of the system are discussed. By combining human worker upper body positions with actions, the perceptual potential of the system is increased, and human–robot collaboration becomes context-aware. Based on the acquired information, the intelligent agent gains the ability to adapt its behavior according to its dynamic and stochastic surroundings.
New combined DT-CWT and HOG descriptor for static and dynamic hand gesture recognition

In recent years, researchers have been focusing on developing Human-Computer Interfaces that are fast, intuitive, and allow direct interaction with the computing environment. One of the most natural ways of communication is hand gestures. In this context, many systems were developed to recognize hand gestures using numerous vision-based techniques, these systems are highly affected by acquisition constraints, such as resolution, noise, lighting condition, hand shape, and pose. To enhance the performance under such constraints, we propose a static and dynamic hand gesture recognition system, which utilizes the Dual-Tree Complex Wavelet Transform to produce an approximation image characterized by less noise and redundancy. Subsequently, the Histogram of Oriented Gradients is applied to the resulting image to extract relevant information and produce a compact features vector. For classification, we compare the performance of three Artificial Neural Networks, namely, MLP, PNN, and RBNN. Random Decision Forest and SVM classifiers are also used to ameliorate the efficiency of our system. Experimental evaluation is performed on four datasets composed of alphabet signs and dynamic gestures. The obtained results demonstrate the efficiency of the combined features, for which the achieved recognition rates were comparable to the state-of-the-art.
Qualitative uncertainty and dynamics of argumentation through dynamic logic

Dynamics and uncertainty are essential features of real-life argumentation, and many recent studies have focused on integrating both aspects into Dung's well-known abstract Argumentation Frameworks (AFs). This paper proposes a combination of the two lines of research through a well-behaved logical tool: Dynamic Logic of Propositional Assignments (DL-PA). Our results show that the main reasoning tasks of virtually all existing formalisms qualitatively representing uncertainty about AFs are encodable in DL-PA. Moreover, the same tool is also useful for capturing dynamic structures, such as control argumentation frameworks, as well as for developing more refined forms of argumentative communication under uncertainty.
Super resolution reconstruction of CT images based on multi-scale attention mechanism

CT diagnosis has been widely used in clinic because of its special diagnostic value. The image resolution of CT imaging system is constrained by X-ray focus size, detector element spacing, reconstruction algorithm and other factors, which makes the generated CT image have some problems, such as low contrast, insufficient high-frequency information, poor perceptual quality and so on. To solve the above problems, a super-resolution reconstruction method of CT image based on multi-scale attention mechanism is proposed. First, use a 3 × 3 and a 1 × 1 convolution layer extracting shallow features. In order to better extract the high-frequency features of CT images and improve the image contrast, a multi-scale attention module is designed to adaptively detect the information of different scales, improve the expression ability of features, integrate the channel attention mechanism and spatial attention mechanism, and pay more attention to important information, retain more valuable information. Finally, sub-pixel convolution is used to improve the resolution of CT image and reconstruct high-resolution CT image. The experimental results show that this method can effectively improve the CT image contrast and suppress the noise. The peak signal-to-noise ratio and structural similarity of the reconstructed CT image are better than the comparison method, and has a good subjective visual effect.
Deep learning: survey of environmental and camera impacts on internet of things images

Internet of Things (IoT) images are captivating growing attention because of their wide range of applications which requires visual analysis to drive automation. However, IoT images are predominantly captured from outdoor environments and thus are inherently impacted by the camera and environmental parameters which can adversely affect corresponding applications. Deep Learning (DL) has been widely adopted in the field of image processing and computer vision and can reduce the impact of these parameters on IoT images. Albeit, there are many DL-based techniques available in the current literature for analyzing and reducing the environmental and camera impacts on IoT images. However, to the best of our knowledge, no survey paper presents state-of-the-art DL-based approaches for this purpose. Motivated by this, for the first time, we present a Systematic Literature Review (SLR) of existing DL techniques available for analyzing and reducing environmental and camera lens impacts on IoT images. As part of this SLR, firstly, we reiterate and highlight the significance of IoT images in their respective applications. Secondly, we describe the DL techniques employed for assessing the environmental and camera lens distortion impacts on IoT images. Thirdly, we illustrate how DL can be effective in reducing the impact of environmental and camera lens distortion in IoT images. Finally, along with the critical reflection on the advantages and limitations of the techniques, we also present ways to address the research challenges of existing techniques and identify some further researches to advance the relevant research areas.
Scalable and Data-Independent Multi-Agent Recommender System Using Social Networks Analysis

Nowadays, many online users find the selection of information and required products challenging due to the growing volume of data on the web. Recommender systems are introduced to deal with information overload. Cold start and data sparsity are the two primary issues in these systems, which lead to a decrease in the efficiency of recommender systems. To solve the problems, this paper proposes a novel method based on social network analysis. Our method leverages a multi-agent system for clustering users and items and predicting relationships between them simultaneously. The information on users and items is extracted from the user-item matrix as distinct graphs. Each of the graphs is then treated as a social network, which is further processed and analyzed by community detection and link prediction procedures. The users are grouped into several clusters by the community detection agent, which results in each cluster as a community. Then link prediction agent identifies the latent relationships between users and items. Simulation results show that the proposed method has significantly improved performance metrics as compared to recent techniques.
Integration of Smart Glasses for Knowledge Transfer in Industrial Remote Maintenance: Learnings from Practice

The use of remote augmented reality (AR) based guidance is seen as an opportunity to address the growing complexity of industrial tasks. In this paper, we look into the question of how remote AR support using smart glasses reduces the maintenance time of complex machinery, and factors that influence their acceptance for productive use. A design science approach was adopted to develop a prototype for a state-of-the-art smart glass for a German small and medium Enterprise (SME) with complex machinery. An evaluation was conducted with 2 industries (one with low and the other with high requirement of AR technology). First, we found that using a remote maintenance application, a time reduction of up to 46% was expected to be achieved. Second, despite its novelty, guidance via AR is accepted better when data privacy concerns are addressed, and when the enterprise has a higher requirement for AR maintenance.
A Context-Aware Knowledge Transferring Strategy for CTC-Based ASR

Non-autoregressive automatic speech recognition (ASR) modeling has received increasing attention recently because of its fast decoding speed and superior performance. Among representatives, methods based on the connectionist temporal classification (CTC) are still a dominating stream. However, the theoretically inherent flaw, the assumption of independence between tokens, creates a performance barrier for the school of works. To mitigate the challenge, we propose a context-aware knowledge transferring strategy, consisting of a knowledge transferring module and a context-aware training strategy, for CTC-based ASR. The former is designed to distill linguistic information from a pre-trained language model, and the latter is framed to modulate the limitations caused by the conditional independence assumption. As a result, a knowledge-injected context-aware CTC-based ASR built upon the wav2vec2.0 is presented in this paper. A series of experiments on the AISHELL-1 and AISHELL-2 datasets demonstrate the effectiveness of the proposed method.
Long term experimental verification of a single chip quantum random number generator fabricated on the InP platform

Abstract This work presents the results from the experimental evaluation of a quantum random number generator circuit over a period of 300 minutes based on a single chip fabricated on the InP platform. The circuit layout contains a gain switched laser diode (LD), followed by a balanced Mach Zehnder Interferometer for proper light power distribution to the two arms of an unbalanced MZI incorporating a 65.4 mm long spiral waveguide that translates the random phase fluctuations to power variations. The LD was gain-switched at 1.3 GHz and the chip delivered a min-entropy of 0.5875 per bit after removal of the classical noise, resulting a total aggregate bit rate of 6.11 Gbps. The recoded data set successfully passed the 15-battery test NIST statistical test suite for all data sets.
CNSNet: A Cleanness-Navigated-Shadow Network for Shadow Removal

The key to shadow removal is recovering the contents of the shadow regions with the guidance of the non-shadow regions. Due to the inadequate long-range modeling, the CNN-based approaches cannot thoroughly investigate the information from the non-shadow regions. To solve this problem, we propose a novel cleanness-navigated-shadow network (CNSNet), with a shadow-oriented adaptive normalization (SOAN) module and a shadow-aware aggregation with transformer (SAAT) module based on the shadow mask. Under the guidance of the shadow mask, the SOAN module formulates the statistics from the non-shadow region and adaptively applies them to the shadow region for region-wise restoration. The SAAT module utilizes the shadow mask to precisely guide the restoration of each shadowed pixel by considering the highly relevant pixels from the shadow-free regions for global pixel-wise restoration. Extensive experiments on three benchmark datasets (ISTD, ISTD+, and SRD) show that our method achieves superior de-shadowing performance.
Signing Avatars - Multimodal Challenges for Text-to-sign Generation

This paper is a positional paper that surveys existing technologies for animating signing avatars from written language. The main grammatical mechanisms of sign languages are described, and in particular the sign inflecting mechanisms in light of the processes of spatialization and iconicity that characterize these visual-gestural languages. The challenges faced by sign language generation systems using signing avatars are then outlined, as well as unresolved issues in building text-to-sign generation systems.