Reviewerly/exHarmony huggingface dataset:
Testing Object Detection for Autonomous Driving Systems via 3D Reconstruction

Object detection is to identify objects from images. In autonomous driving systems, object detection serves as an intermediate module, which is used as the input of autonomous decisions for vehicles. That is, the accuracy of autonomous decisions relies on the object detection. The state-of-the-art object detection modules are designed based on Deep Neural Networks (DNNs). It is difficult to employ white-box testing on DNNs since the output of a single neuron is inexplicable. Existing work conducted metamorphic testing for object detection via image synthesis: the detected object in the original image should be detected in the new synthetic image. However, a synthetic image may not look real from humans' perspective. Even the object detection module fails in detecting such synthetic image, the failure may not reflect the ability of object detection. In this paper, we propose an automatic approach to testing object detection via 3D reconstruction of vehicles in real photos. The 3D reconstruction is developed via vanishing point estimation in photos and heuristic based image insertion. Our approach adds new objects to blank spaces in photos to synthesize images. For example, a new vehicle can be added to a photo of a road and vehicles. In this approach, the output synthetic images are expected to be more natural-looking than randomly synthesizing images. The experiment is conducting on 500 driving photos from the Apollo autonomous driving dataset.
Signed Diffie-Hellman Key Exchange with Tight Security

We propose the first tight security proof for the ordinary two-message signed Diffie-Hellman key exchange protocol in the random oracle model. Our proof is based on the strong computational Diffie-Hellman assumption and the multi-user security of a digital signature scheme. With our security proof, the signed DH protocol can be deployed with optimal parameters, independent of the number of users or sessions, without the need to compensate any security loss. We abstract our approach with a new notion called verifiable key exchange.
360° Virtual Tour of the Traditional Malay House as an Effort for Cultural Heritage Preservation

Abstract The rapid growth in technology has created new innovation in the field of information technology. One of the innovations created from technological advancement is virtual reality that can be extended through virtual tour that provides real-time experience in exploring a space or an area. Nevertheless, preliminary research on the development of digital sector in heritage architecture only focused on historical site instead of building. Although there is an exploration paid to the heritage building, the effort is limited to colonial buildings and little attention has been given to the local heritage architecture. Hence, this paper aims to explore the development of 360 ° virtual tour of the traditional Malay house that is over 100 years old as one of an important measure to preserve, protect and interpret the built heritage and cultural history. The primary data was analysed from the building inventory through high-resolution photography using a fisheye lens digital camera. Photos collected were combined to create panoramic effect using a stitching technique. The 360 ° virtual tour application was developed based on the Multimedia Development Life Cycle theory. The findings show the application is produced as a digital database consisting of physical qualitative attributes data for university’s management and recording for posterity. The outcome from this study has proved how technology converts the traditional learning and museum visit can become an enjoyable platform of learning, motivating and understanding of building structures and unique architectural details.
Analyzing Visual Perception and Predicting Locomotion using Virtual Reality and Eye Tracking

Locomotion and vison are closely linked.When users explore virtual environments by walking they rely on stable visible landmarks to plan and execute their next movement.In my research I am developing novel methods to predict locomotion paths of human subjects for the immediate future, i.e. the next few seconds.I aim to connect different types of behavioral data (eye, hand, feet and head tracking) and test their reliability and validity for predicting walking behavior in virtual reality.Such a prediction will be very valuable for natural interaction, for example in redirected walking schemes.My approach begins with an evaluation of the quality of data gathered with current tracking methods.Informative experimental conditions need to be developed to find meaningful patterns in natural walking.Next, raw tracked data of different modalities need to be connected with each other and aggregated in a useful way.Thereafter, possible valid predictors need to be developed and compared to already functioning predicting algorithms (e.g.[2,6,12]).As a final goal, all valid predictors shall be used to create a prediction algorithm returning the most likely future path when exploring virtual environments.
Ghost-Free Deep High-Dynamic-Range Imaging Using Focus Pixels for Complex Motion Scenes

Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.
We Can Do More to Save Guqin: Design and Evaluate Interactive Systems to Make Guqin More Accessible to the General Public

Guqin is a plucked seven-string traditional Chinese musical instrument that exists for over 3,000 years. However, as an Intangible World Cultural Heritage, the inheritance of Guqin and its culture in modern society is in deep danger. According to our study with 1,006 Chinese worldwide, Guqin as an instrument is not well-known and barely accessible. To better promote Guqin, we developed two interactive systems: VirGuqin and MRGuqin. VirGuqin was developed using a low-cost motion tracking device and was tested in a museum. 89% of 308 participants expressed an increase in interest in learning Guqin after using our system. MRGuqin was developed as a mixed reality learning environment to reduce the entry barrier to Guqin, and was tested by 16 participants, allowing them to learn Guqin significantly faster and perform better than the current practice. Our study demonstrates how technology can be used to help the inheritance of this dying art.
Developing a LBPH-based Face Recognition System for Visually Impaired People

A large number of people around the world are suffering from visual impairment which is a global health issue. These visually challenged people face a great deal of difficulties in carrying out their day-to-day activities. Recognizing a person is one of the major problems faced by them. This document represents a face recognition system with auditory output which can be beneficial for visually challenged people in recognizing known and unknown persons. Proposed face recognition system is comprised of three main modules including dataset creation, dataset training, and face recognition. Here, Haar Cascade Classifier is used to detect face from a live video stream and then Local Binary Pattern Histogram (LBPH) algorithm is applied to create the recognizer for face recognition using OpenCV-Python library. This system can detect and recognize multiple people and is also capable of recognizing from both front and side face. The overall face recognition accuracy is about 93%. Apart from visually challenged people, old people with Alzheimer's disease can also be benefited using this system.
Spectral Domain Convolutional Neural Network

The memory consumption of most Convolutional Neural Network (CNN) architectures grows rapidly with increasing depth of the network, which is a major constraint for efficient network training on modern GPUs with limited memory, embedded systems, and mobile devices. Several studies show that the feature maps (as generated after the convolutional layers) are the main bottleneck in this memory problem. Often, these feature maps mimic natural photographs in the sense that their energy is concentrated in the spectral domain. Although embedding CNN architectures in the spectral do-main is widely exploited to accelerate the training process, we demonstrate that it is also possible to use the spectral domain to reduce the memory footprint, a method we call Spectral Domain Convolutional Neural Network (SpecNet) that performs both the convolution and the activation operations in the spectral domain. The performance of SpecNet is evaluated on three competitive object recognition benchmark tasks (CIFAR-10, SVHN, and ImageNet), and compared with several state-of-the-art implementations. Overall, SpecNet is able to reduce memory consumption by about 60% without significant loss of performance for all tested networks.
Multiple Perspective Caption Generation with Attention Mechanism

In caption generation, a caption generation system generates a caption, which describes the content of the image with natural language and needs to understand both an image and a text. So caption generation is an essential task in natural language processing and image processing. Many researchers recently pay attention to deep learning as a key technique to construct the caption generation system because deep learning can construct an intermediate representation, which is shared in both image processing and natural language processing. First, the system generates a feature from a given image with convolutional neural networks. Eventually, the system generates a word sequence, a caption, from the feature. It means that the system consists of two modules, the image processing module and the language model module and both of the modules are simultaneously trained with a training dataset. The deep learning based caption generation system is a blackbox system and it is difficult to collaborate with a human. So, we introduce attention mechanism in the caption generation system and we control caption generation by the attention weights. The usual caption generation systems can generate only a single caption from one image because the system can generate a caption from an image directly. The proposed system can generate some different captions form the same image because we can control the initial attention weights. Our results demonstrate how we collaborate with deep learning and the fact that the collaboration improves caption generation.
Cameron-Liebler line classes with parameter <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si1.svg"><mml:mi>x</mml:mi><mml:mo linebreak="goodbreak" linebreakstyle="after">=</mml:mo><mml:mfrac><mml:mrow><mml:msup><mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo linebreak="badbreak" linebreakstyle="after">+</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup></mml:mrow><mml:mrow><mml:mn>3…

Cameron-Liebler line classes were introduced in [5], and motivated by a question about orbits of collineation groups of PG(3,q). These line classes have appeared in different contexts under disguised names such as Boolean degree one functions, regular codes of covering radius one, and tight sets. In this paper we construct an infinite family of Cameron-Liebler line classes in PG(3,q) with new parameter x=(q+1)2/3 for all prime powers q congruent to 2 modulo 3. The examples obtained when q is an odd power of two represent the first infinite family of Cameron-Liebler line classes in PG(3,q), q even.
Research on Human Target Detection and Tracking Based on Artificial Intelligence Vision

With the rapid development of multimedia technology and the continuous improvement of computer performance, digital video monitoring system has gradually replaced the traditional analog monitoring system and has been widely used in public places such as airports, stations, banks, hotels and so on. As an important part of intelligent video surveillance system, target detection and target tracking technology has become an important research direction in the field of vision. Human behavior detection and recognition based on artificial intelligence vision has developed rapidly due to the increasing demand, and has become a hot spot in the research field of computer vision and artificial intelligence. Under complex background, the detection, recognition and automatic tracking of human moving objects based on artificial intelligence vision are the key and difficult points in the field of computer vision research. In this paper, the image preprocessing, detection and extraction, matching recognition and automatic tracking of human moving targets based on artificial intelligence vision in complex environment are analyzed and studied, which lays a good foundation for the research and implementation of human target vision monitoring system.
Enhancing Image Steganography Via Stego Generation And Selection

Unlike most existing steganography methods which are mainly focused on designing embedding cost, in this paper, we propose a new method to enhance existing steganographic methods via stego generation and selection. The proposed method firstly trains a steganalytic network according to the steganography to be enhanced, and then tries to adjust a tiny part of original embedding costs based on the magnitudes of it and the corresponding gradients obtained from the pre-trained network, and generates many candidate stegos in a random manner. Finally, the method selects a stego according to its image residual distance to cover. Extensive experimental results have shown that the proposed method can siginficantly enhance the security performance of current steganography in spatial domain against four steganalytic classifiers. In addition, comparative analysis between original stegos and the resulting ones with the proposed method are given.
Connectionist temporal classification loss for vector quantized variational autoencoder in zero-shot voice conversion

Vector quantized variational autoencoder (VQ-VAE) has recently become an increasingly popular method in non-parallel zero-shot voice conversion (VC). The reason behind is that VQ-VAE is capable of disentangling the content and the speaker representations from the speech by using a content encoder and a speaker encoder, which is suitable for the VC task that makes the speech of a source speaker sound like the speech of the target speaker without changing the linguistic content. However, the converted speech is not satisfying because it is difficult to disentangle the pure content representations from the acoustic features due to the lack of linguistic supervision for the content encoder. To address this issue, under the framework of VQ-VAE, connectionist temporal classification (CTC) loss is proposed to guide the content encoder to learn pure content representations by using an auxiliary network. Based on the fact that the CTC loss is not affected by the sequence length of the output of the content encoder, adding the linguistic supervision to the content encoder can be much easier. This non-parallel many-to-many voice conversion model is named as CTC-VQ-VAE. VC experiments on the CMU ARCTIC and VCTK corpus are carried out to evaluate the proposed method. Both the objective and the subjective results show that the proposed approach significantly improves the speech quality and speaker similarity of the converted speech, compared with the traditional VQ-VAE method.
Multimodal Fusion Using Sparse Cca For Breast Cancer Survival Prediction

Effective understanding of a disease such as cancer requires fusing multiple sources of information captured across physical scales by multimodal data. In this work, we propose a novel feature embedding module that derives from canonical correlation analyses to account for intra-modality and inter-modality correlations. Experiments on simulated and real data demonstrate how our proposed module can learn well-correlated multi-dimensional embeddings. These embeddings perform competitively on one-year survival classification of TCGA-BRCA breast cancer patients, yielding average F1 scores up to 58.69% under 5-fold cross-validation.
Menzerath’s Law in the Syntax of Languages Compared with Random Sentences

The Menzerath law is considered to show an aspect of the complexity underlying natural language. This law suggests that, for a linguistic unit, the size (y) of a linguistic construct decreases as the number (x) of constructs in the unit increases. This article investigates this property syntactically, with x as the number of constituents modifying the main predicate of a sentence and y as the size of those constituents in terms of the number of words. Following previous articles that demonstrated that the Menzerath property held for dependency corpora, such as in Czech and Ukrainian, this article first examines how well the property applies across languages by using the entire Universal Dependency dataset ver. 2.3, including 76 languages over 129 corpora and the Penn Treebank (PTB). The results show that the law holds reasonably well for x&gt;2. Then, for comparison, the property is investigated with syntactically randomized sentences generated from the PTB. These results show that the property is almost reproducible even from simple random data. Further analysis of the property highlights more detailed characteristics of natural language.
Gated fusion network for SAO filter and inter frame prediction in Versatile Video Coding

In order to achieve higher coding efficiency, the Versatile Video Coding (VVC) standard includes several new components at the expense of an increase in decoder computational complexity. These technologies often create ringing and contouring effects on the reconstructed frames at a low bit rate and introduce blurring and distortion. To smooth those visual artifacts, the H.266/VVC framework supports four post-processing filter operations. The state-of-the-art CNN-based in-loop filters prefer to deploy multiple networks for various quantization parameters and frame resolutions, which increases training resources and subsequently becomes overhead at decoder frame reconstruction. This paper presents a single deep-learning-based model for sample adaptive off-set (SAO) non-linear filtering operation on the decoder side, uses feature correlation among adjacent frames, and substantiates the merits of intra–inter frame quality enhancement. We introduced a variable filter size dual multi-scale convolutional neural network (D-MSCNN) to attenuate the compression artifact and incorporated strided deconvolution to restore the high-frequency details on the distorted frame. Our model follows sequential training across all QP values and updates the model weights. Using data augmentation, weight fusion, and residual learning, we demonstrated that our model could be trained effectively by transferring the convolution prior feature indices to the decoder to produce a dense output map. The Objective measurements demonstrate that the proposed method outperforms the baseline VVC method in PSNR, MS-SSIM, and VMAF metrics and achieves an average of 5.16% bit rate saving on different test sequence categories.
Sequence-to-Sequence Learning for Indonesian Automatic Question Generator

Automatic question generation is defined as the task of automating the creation of question given a various of textual data. Research in automatic question generator (AQG) has been conducted for more than 10 years, mainly focused on factoid question. In all these studies, the state-of-the-art is attained using sequence-to-sequence approach. However, AQG system for Indonesian has not ever been researched intensely. In this work we construct an Indonesian automatic question generator, adapting the architecture from some previous works [1] [2]. In summary, we used sequence-to-sequence approach using BiGRU, BiLSTM, and Transformer with additional linguistic features, copy mechanism, and coverage mechanism. Since there is no public large and popular Indonesian dataset for question generation, we translated SQuAD v2.0 factoid question answering dataset, with additional Indonesian TyDiQA dev set for testing. The system achieved BLEU1, BLEU2, BLEU3, BLEU4, and ROUGE-L score at 38.35, 20.96, 10.68, 5.78, and 43.4 for SQuAD, and 40.01, 20.68, 10.28, 6.44, 44.17 for TyDiQA, respectively. We found that the system performed well when the expected answers are named entities and are syntactically close with the context explaining them. Additionally, from native Indonesian perspective, the best questions generated by our best models on their best cases are acceptable and reasonably useful.