Reviewerly/exHarmony huggingface dataset:
Accelerate Three-Dimensional Generative Adversarial Networks Using Fast Algorithm

Three-dimensional generative adversarial networks (3D-GAN) have attracted widespread attention in three-dimension (3D) visual tasks. 3D deconvolution (DeConv), as an important computation of 3D-GAN, significantly increases computational complexity compared with 2D DeConv. 3D DeConv has become a bottleneck for the acceleration of 3D-GAN. Previous accelerators suffer from several problems, such as large memory requirements and resource underutilization. To handle the above issues, a fast algorithm for 3D DeConv (F3DC) is proposed in this paper. F3DC applies a fast algorithm to reduce the number of multiplications and achieves a significant algorithmic strength reduction. Besides, F3DC removes the extra memory requirement for overlapped partial sums and avoids computational imbalance to fully utilize resources. Moreover, we design an F3DC-based hardware architecture, which consists of four fast processing units (FPUs). Each FPU includes a pre-process module, a EWMM module and a post-process module for F3DC transformation. By implementing our design on the Xilinx VC709 platform for 3D-GAN, we achieve a throughput up to 1700 GOPS and 4× computational efficiency improvement compared with prior works.
Effective Data Augmentation Methods for CCMT 2022

The purpose of this paper is to introduce the specific situation in which Guangxi University participated in the 18th China Conference on Machine Translation (CCMT 2022) evaluation tasks. We submitted the results of two bilingual machine translation (MT) evaluation tasks in CCMT 2022. One is Chinese-English bilingual MT tasks from the news field, the other is Chinese-Thai bilingual MT tasks in low resource languages. Our system is based on Transformer model with several effective data augmentation strategies which are adopted to improve the quality of translation. Experiments show that data augmentation methods have a good impact on the baseline system and aim to enhance the robustness of the model.
Mobility Data Stories for a Better Understanding of Mobility Data

Numerous stakeholders are involved in mobility planning, such as citizens’ initiatives, political decision-makers, and mobility consultants. Here, data and its linkage form an important building block for decision-making. However, the challenge is to process heterogeneous data sources in a problem-adequate and target group-specific manner. Therefore, this publication addresses how decision-relevant contexts can be visualized and developed into mobility data stories. First, the approaches and principles of Neifer, Freytag, and Field are presented. They show that data stories are formed from an intersection of the key components data, visualizations, and stories. After that, an example is used to show how Mobility Data Stories are developed, starting from the core question, through the selection of data, the conception of the story, to the visualization. The sample solution developed in the research process is then evaluated regarding usability and user experience with the stakeholders. Based on the evaluation results, a method for creating target-group-specific Mobility Data Stories is derived to support the stakeholders in the mobility sector with the visual and content-related preparation of the data. Further, the target-oriented mobility planning process is structured regarding the potential of mobility data stories. Finally, general guidelines for creating target-group-specific Mobility Data Stories are derived. The results show the potential of Mobility Data Stories.
Hierarchical Parallel Vision Processor for High-Speed Ship Detection

Ship detection is essential in ship rescue and marine traffic safety. However, high-speed real-time ship detection has become problematic in practical applications. In this brief, we propose a high-performance ship detection system utilizing the co-design of algorithm and hardware. First, the proposed Coarse-to-Fine Classification and Segmentation algorithm contains a two-stage convolutional neural network. It quickly locates the Region of Interest (RoI) and then accurately locates the ship’s position in the RoI without low-parallelism non-maximum suppression (NMS) post-processing. Second, we proposed a Hierarchical Parallel Vision Processor, including a pixel-parallel processing unit, a patch-parallel processing unit, and a global micro-processing unit, ensuring high-speed processing with different parallelism. Finally, we designed a high-speed real-time ship detection system. The experimental results show that our ship detection system based on the frequency of 100Mhz on Arria 10 FPGA can achieve a speed of 4778FPS, which is beneficial in practical applications.
Moving in a Simulated Environment Through Deep Reinforcement Learning

Reinforcement learning is a field of artificial intelligence that is continuously evolving and has a wide variety of applications. In recent years major progress has been made in the application of deep reinforcement learning to high-dimensional problems with continuous state and action spaces. This paper presents a complete analysis of the application of the soft actor-critic algorithm to teach a four legged robot with three joints on each leg how to move towards the center of a virtually simulated environment. The general formulation of the reinforcement learning problem is first presented, followed by the description of the environment under analysis and the applied algorithm. Afterwards, the obtained results are compared against those of a manually programmed policy, closing with a discussion of some key design choices and common challenges.
Totems: Physical Objects for Verifying Visual Integrity

We introduce a new approach to image forensics: placing physical refractive objects, which we call totems, into a scene so as to protect any photograph taken of that scene. Totems bend and redirect light rays, thus providing multiple, albeit distorted, views of the scene within a single image. A defender can use these distorted totem pixels to detect if an image has been manipulated. Our approach unscrambles the light rays passing through the totems by estimating their positions in the scene and using their known geometric and material properties. To verify a totem-protected image, we detect inconsistencies between the scene reconstructed from totem viewpoints and the scene's appearance from the camera viewpoint. Such an approach makes the adversarial manipulation task more difficult, as the adversary must modify both the totem and image pixels in a geometrically consistent manner without knowing the physical properties of the totem. Unlike prior learning-based approaches, our method does not require training on datasets of specific manipulations, and instead uses physical properties of the scene and camera to solve the forensics problem.
A virtual reality research of Gait analysis in the medicine fields

Therapeutic effects for rehabilitation of the older adult are provided by virtual realities therapy, but the body’s balance remains unclear. The objective was to characterize the effect of senior physiotherapy therapy treatments with VRT on their balance-rehabilitation. Virtual Reality is a technology often used in many fields of cinema, sport, entertainment and simulation. Machine imaging and materials seek in the present circumstances to simulate a real presence. Their sensory abilities in medicine are of practical value. His significance in the field of healthcare should thus be explored. Older individuals are most frequently affected by injury and injury-related deaths. This study evaluated the effect on the balance and the gait capabilities of falling elderly individuals. Ten articles met the inclusion criteria, which demonstrated variety in treatment types (70%) and evaluated outcomes (60 percent). The article focuses on a review of literature in which we comprehend how this technology solves a medical issue to save the lives and vital applications of a patient.
Design of Vocabulary Query System in Computer Aided English Translation Teaching

From “Internet” to “Internet +”, we have entered the era of “integration 2.0”. In the era of “integration 2.0”, media integration has entered the terminal to win by competing for channels. This paper studies the design and style of online game passing through the Internet plus, and draws the conclusion that the artistic performance and entertainment value of online games are mainly based on the interaction between online game player and its entertainment, and then the game is gradually developed according to the development of the story of the game. At the same time, from the perspective of players, this paper makes a detailed analysis of the sense of time and space and the world outlook of the game, so as to find the method of game clearance scene design and creation. We all know that the story planning of the game comes from its script, and the whole game is composed of emerging contradictions and conflicts. The development of one contradiction leads to the beginning of the next contradiction, which is superimposed, sublimated and cut into the theme. The reason why players can immersively participate in the game and indulge in it is mostly affected by the player’s visual psychology and scene design style. It is a necessary means to enhance the player’s interest and plays an important role in the game scene design.
Multiresolution Neural Networks for Imaging

We present MR-Net, a general architecture for multiresolution neural networks, and a framework for imaging applications based on this architecture. Our coordinate-based networks are continuous both in space and in scale as they are composed of multiple stages that progressively add finer details. Besides that, they are a compact and efficient representation. We show examples of multiresolution image representation and applications to texture magnification, minification, and antialiasing.
Analyzing GCN Aggregation on GPU

Graph convolutional neural networks (GCNs) are emerging neural networks for graph structures that include large features associated with each vertex. The operations of GCN can be divided into two phases - aggregation and combination. While the combination just performs matrix multiplications using trained weights and aggregated features, the aggregation phase requires graph traversal to collect features from adjacent vertices. Even though neural network applications rely on GPU’s massively parallel processing, GCN aggregation kernels exhibit rather low performance since graph processing using compressed graph structures provokes frequent irregular accesses in GPUs. In order to investigate the performance hurdles of GCN aggregation on GPU, we perform an in-depth analysis of the aggregation kernels using real GPU hardware and a cycle-accurate GPU simulator. We first analyze the characteristics of the popular graph datasets used for GCN studies. We reveal the fractions of non-zero elements in feature vectors are diverse among datasets. Based on the observation, we build two types of aggregation kernels that handle uncompressed and compressed feature vectors. Our evaluation exhibits the performance of aggregation can be significantly influenced by kernel design approaches and feature density. We also analyze the individual loads that access the data arrays of the aggregation kernels to specify critical loads. Our analysis reveals the performance of GPU memory hierarchy is influenced by access patterns and feature size of graph datasets. Based on our observations we discuss possible kernel design approaches and architectural ideas that can improve the performance of GCN aggregation.
Learning a Compact Spatial-Angular Representation for Light Field

The recent emergence of light field technology has led to new opportunities for immersive visual communication that has a need for high spatial and angular resolution, both of which contribute to a large image storage footprint and high-latency transmission. Task-driven downsampling methods have been proposed as a solution, and have shown improvements in single-image restoration. However, they are inevitable to disregard light field's intrinsic properties in the corresponding tasks. In this paper, we propose a light-field-specific task-driven downsampling framework, called the LFCrNet. The LFCrNet operates on a learning-based decreasing and increasing resolution in an end-to-end manner in order to utilize a cross-view asymmetric sampling technique. In detail, it separates raw data into disparity and non-disparity patterns by measuring pixel-wise residuals between the sub-aperture central view and auxiliary views. Then, a chain of 3-D deformable residual blocks (DRBs) is used to extract disparity features and manage these features regard of their intrinsic property individually. Afterwards, they are compacted into spatio-angular domains through a 3-D deformable downsampler (3-DDS). The non-disparity information is integrated into a separate pipeline that leverages spatial similarity across multiple light field views. This technique is capable of preserving specific occlusion components, and subsequently, restoring them using a learning-based upscaling method to generate a high-quality reconstruction. In general, our method has shown superior performance on multiple open-source datasets by a significant margin.
The architecture design of training data microservice based on blockchain technology

In order to build a data-driven service system and improve the scientific level of the training department's work efficiency, we adopt the concept of microservice development. Utilizing the distributed, fault-tolerant, and sharing characteristics, we apply blockchain technology to the back-end data system and deploy a lightweight front-end sharing architecture, realizing the design concept of training data microservice model. In this paper, we start with the actual needs of combining the characteristics of blockchain technology and training data, focusing on analyzing the application logic and applicable scenarios of blockchain technology in training data microservice. We propose the concept of credible data sharing service for on-chain storage and off-chain transmission, and designed a blockchain-based database access control mechanism. Finally, we propose the experiment and the conclusion to the application system development method of the service.
Deidentification of Drivers’ Face Videos: Scope and Challenges in Human Factors Research

Data sharing across disciplines helps to build collaboration, and advance research. With recent development in data-driven models, there is an unprecedented need for data. However, data collected from human research subjects are required to follow proper ethical guidelines. Researchers have an obligation to protect the privacy of research participants and address ethical and safety concerns when data contains personally identifying information (PII). This paper addresses this problem with a focus on sharing drivers’ face videos for transportation research. The paper first gives an overview of the multitude of problems that are associated with sharing drivers’ videos. Then it demonstrates the possible directions for data sharing by de-identifying drivers’ faces using artificial intelligence-based techniques. The results achieved through the proposed techniques were evaluated qualitatively and quantitatively to prove the validity of the suggested methods. We specifically demonstrated how face-swapping algorithms can effectively de-identify faces while still preserving important attributes related to human factor research including eye movements, head movements, mouth movements, etc. Finally, we discuss possible measures to share such de-identified videos with the greater research community.
A Survey on Generative Adversarial Network Based Person Re-Identification Method

In recent years, with the growing demand for public safety and the rapid development of intelligent surveillance networks, person re-identification (Re-ID) has become one of the hottest research topics in the field of computer vision, which aims at retrieving a person of interest through multiple cameras with non-overlapping views. First, the current classical person Re-ID datasets are introduced. Second, the latest GAN-based person Re-ID methods are focused on sorting out. These methods are classified into three categories: image-image style transfer, data enhancement and invariant feature learning according to the characteristics and application scenarios of GAN. And basic principles, advantages, and disadvantages are summarized for each category. Then, the states of the art of the classic algorithms mentioned are compared on the mainstream datasets. Finally, the present challenging problems of person Re-ID and directions for future research are discussed.
An Insight into the Reusability of Stack Overflow Code Fragments in Mobile Applications

Code reusing from crowd-sourced sites, specifically Stack Overflow (SO), is a widely practiced fundamental phenomenon in software development. However, severe consequences might arise if the incorporated SO code snippets contain code smells that cause software inconsistency. In particular, software bugs and failures cost trillions of dollars every year, including severe fatalities. This study reveals the impact of SO code fragments reused in mobile application codebase regarding software bugs and imminent maintenance. Additionally, we perform an intense analysis on reused code snippets (SO) and other (nonSO) code snippets for ten open-source and industrial projects. Later, we also investigate the regular properties of SO answers (i.e., answer scores and answerer reputation) to know whether those properties help identify appropriate SO fragments when reused, considering that 2.1 million SO code snippets are written in Java. The analysis exhibits, 1) the proportion of reused SO code is comparatively higher in industrial mobile apps than in open-source; 2) SO code fragments are significantly more change-prone than non-SO code; 3) SO code snippets are responsible for bug occurrence in later revisions, which is comparatively higher in industrial projects than in open-source ones. Besides, the SO code snippets cannot be judged whether it is buggy or not using the regular properties (i.e., answer scores and answerer reputation). Our experimental results can assist the SO, research, and mobile developer communities in strengthening re-usability concerns to facilitate code-quality improvement and minimize software bugs due to SO code.
A Synchronized Multi-view System for Real-Time 3D Hand Pose Estimation

In this demo, we present a high-speed and high-quality synchronous multi-view system, which has extremely low latency ( $$10^{-8}$$ s) benefiting the high-fidelity performance of 3D hand pose estimation. Our synchronous multi-view system adopts the unified trigger signal generated by the newly designed synchronous signal generator to control all cameras to work at the same time. It can not only be used for real-time hand pose estimation but also for other moving targets, e.g., body, cloth, general objects, etc. With the help of the developed multi-view system, we have achieved real-time 3D hand pose estimation results.
Behavior Recognition Network for Small Target Vehicle based on Attention Mechanism

Vehicle behavior recognition is one of the most important research fields in intelligent transportation. In the field of traffic surveillance videos, target vehicles only occupy small proportion of the video frame,making it difficult for the network to extract key features of small target vehicles. To solve this problem, this paper constructed vehicle-3C, a video dataset for vehicle behavior recognition, and was combined with deep learning theory. As a result, it proposed a vehicle behavior recognition method based on the theory of Two-Stream with Attention Network (TSAN). For this method, the two-stream convolutional network is used as a basic framework and attention units are embedded in it to extract temporal (motion) features and spatial features of target vehicles in traffic surveillance videos, and then the temporal and spatial features are fused for category judgment. The results showed that TSAN could achieve 81.8% identification accuracy in vehicle-3C dataset, which was better than other behavior recognition methods based on deep learning. In addition, TSAN also achieves a recognition accuracy of 77.2% in dataset UCF-101, which verifies the generalization performance of the network.
An Orientation-Aware Anchor-Free Detector for Aerial Object Detection

Most aerial object detectors mainly adopt anchor-based methods, which require manually designed rough anchors. However, these anchors usually contain background redundancy unrelated to the objects, decreasing detection performance and introducing additional calculations. In this paper, we propose an orientation-aware anchor-free network (OAF-Net) for object detection. OAF-Net first produces coarse oriented boxes by coarse location network based on pixel-level regression and then refines them with aligned features. In particular, we also apply a new metric to measure the orientation-aware center-ness, which is a weighting strategy of positive samples to reduce the different contributions of the low-quality detection boxes. Experiments on the DOTA1.0 and HRSC2016 datasets reveal that OAF-Net outperforms the state-of-the-art in terms of comprehensive detection performance for various objects with different scales and orientations in aerial images.
IARG: Improved Actor Relation Graph Based Group Activity Recognition

Video understanding is to recognize and classify different actions or activities appearing in the video. A lot of previous work, such as video captioning, has shown promising performance in producing general video understanding. However, it is still challenging to generate a fine-grained description of human actions and their interactions using state-of-the-art video captioning techniques. The detailed description of human actions and group activities is essential information, which can be used in real-time CCTV video surveillance, health care, sports video analysis, etc. This study proposes a video understanding method that mainly focused on group activity recognition by learning the pair-wise actor appearance similarity and actor positions. We propose to use Normalized cross-correlation (NCC) and the sum of absolute differences (SAD) to calculate the pair-wise appearance similarity and build the actor relationship graph to allow the graph convolution network to learn how to classify group activities. We also propose to use MobileNet as the backbone to extract features from each video frame. A visualization model is further introduced to visualize each input video frame with predicted bounding boxes on each human object and predict individual action and collective activity.
Research on Variational Autoencoders and its Interpretability Based on Gaussian Cloud Model

As one of the important generative models, Variational Autoencoders (VAEs) can complete the feature characterization and generation of data. However, their reconstruction effect on high-resolution images is often not high. The poor image quality is partly due to the limited space for hiding variables. In order to solve this problem, based on the optimization model GCMVAE based on VAEs, this paper improves the generation ability of the model and studies its interpretability. In this study, the interpretability research focuses on the interpretation of some models without semantic information in different stages of the model. Finally, through the analysis of the results, we can get the characteristics of data, the distribution characteristics of implicit variables and the different learning stages experienced in the process of data generation.