Reviewerly/exHarmony huggingface dataset:
Code-switched Text Data Augmentation for Chinese-English Mixed Speech Recognition

Code-switching is a common mode of language expression, which means that two or more languages are used interchangeably in a conversation. At present, the development of such code-switching technology in the field of speech recognition research is still limited by insufficient training corpus of text, which affects the system performance. This paper will use a neural network to train a generator to generate code-switching text to expand the corpus to achieve the purpose of improving the mixed recognition rate of Chinese and English. Our method is to use the Chinese and English texts in the SEAME corpus to train the BERT-BiLSTM-CRF model and use the model to know the code-switching position, generating sentences that conform to the characteristics of this corpus. The experimental results show that the method in this paper has better performance than other methods.
An Approach for Domain Relation Extraction

Extraction of relations among entities from texts is critical for domain knowledge representation. In this paper, an association graph was constructed to represent the dependencies among entities and relations, upon which a relation extraction method was proposed to predict the relations in domain texts. Experimental results on various annotated domain datasets demonstrate that the recall of our proposed method outperforms the other relation extraction models.
A Consideration on Efficient Detection Method of Anormal Responses in High-dimensional Questionnaire Data

We have been studying to detect anormal responses in high-dimensional questionnaire data, that may affect the overall analysis results and are to be removed in the preprocess, more efficiently. In this paper, we apply principal component analysis (PCA) and multiple correspondence analysis (MCA) as dimension reduction methods, and x-means and gaussian mixture model (GMM) as clustering algorithms to the high-dimensional questionnaire data. Then, we examine the combinations of these methods for detecting anormal responses that are significantly far from the cluster centers or the distribution centers. Also, we employ principal component pursuit (PCP), where the absolute value sum for each response in the sparse matrix is used as anormal score to directly detect anormal responses. As a result, we find both of MCA+x-means and PCP achieve to detect reasonable anormal responses with shorter execution time.
Automatic Reference Image Selection Approach for Reference-Based Image Super-Resolution

Super-resolution (SR) technology can be used for increasing image resolution. Therefore, deep learning-based SR was developed around single-image SR, which requires only one image as input. Thus, reference-based image SR, which uses a high-resolution reference image for SR, was invented. The high-resolution reference image provides more accurate correction. However, since the accuracy of reference-based image SR depends on the reference image itself, selecting an appropriate reference image is a vital and challenging task. This study demonstrates that reference-based image SR is superior to single-image SR by comparing the two, and verifies that the accuracy depends on the reference image. Further, an automatic reference image selection network is proposed to combat the issue of selecting appropriate reference images for reference-based image SR.
A Similarity-Based Sensor Data Recovery Scheme

Currently, one of the main problems in sensor data delivery is handling data loss. And retransmission is a basic solution for data recovery. However, existing recovery schemes do not consider retransmission reduction based on data similarity. Hence, in this paper, we propose a similarity-based sensor data recovery method. Through simulation, our results show that the proposed method achieves retransmission reduction in terms of bandwidth consumption and network load.
The Boundary Identification Algorithm for a Process Control

The paper deals with an algorithm for the limit point determination of the parameter space region (cluster boundaries, domain boundaries, etc.). The algorithm uses a fixed point and a direction vector. The algorithm uses the representation of data analysis results in a multidimensional parameter space. The data representation corresponds to the model considered in author's previous works. The paper provides the proof of the algorithm's correctness, its comparison with an alternative approach and the calculation of computational complexity. The algorithm is developed to be used in automated process control in two cases. The first case when it is required to keep the object parameters within certain limits. The second case when it is required to find the optimal trajectory in the parameter space.
A Reversible Data Hiding Method in Compressible Encrypted Domain for OpenEXR Images

This paper proposes a reversible data hiding method in compressible encrypted-OpenEXR images. Our method encrypts a target image with an encryption-then-compression (EtC) method and embeds a payload with a prediction-error expansion with histogram shifting (PEE-HS) method. Our method has achieved high compression efficiency by using the EtC method. Additionally, we can decrypt the marked encrypted images without data extraction. The marked images have high quality since we have embedded the payload into mantissa bits. We evaluated our method in terms of the compression efficiency, marked image quality using logPSNR, and hiding rate.
Evaluation of Linguistic Properties of Synthesized Laughter Using its Transcription

Recent speech synthesis has made it possible to synthesize speech that is indistinguishable from human speech. Its use is not limited to the narration of videos, and speech interfaces for appliances, but has also emerged in scenarios where it interacts with humans, such as dialogue assistants. Therefore, synthesizing and evaluating more human-like and emotional speech are becoming increasingly important. In this study, laughter was generated from an emotionless speech by voice quality conversion, and its transcriptions were given by 9 subjects. We attempt to quantify the quality of synthesized speech using the edit distance of the transcriptions, which measures how linguistically consistent the given transcriptions were among the subjects. We also measured the Mean Opinion Score (MOS) of the laughter speech using the same subjects.The results showed that the linguistic information acquired by the subjects was close to that of natural laughter, even when the voice quality transformation model was trained without any special linguistic processing. However, there seems to be no strong correlation between the quality of the linguistic information given by human subjects and the subjective quality.
Accurate and Efficient Generation of High-Resolution Facial Expression Images by Multi-Task Learning Using Generative Adversarial Networks

Recently, the means to see human face images have increased owing to the spread of smartphones and social networking services. Especially, in the field of face images, the generation of face images using facial expression transformation has already been realized using deep learning-based approaches. However, in the existing deep learning-based models, only low-resolution images can be generated owing to limited computational resources, and the generated images are blur or aliasing. To solve this problem, we proposed to enhance the resolution of the generated facial image by using a super resolution network after the generative model, which can be considered as a serial model in our previous work. In this paper, we newly proposed a parallel model, which train the generative adversarial network and the super-resolution network through multi-task learning. Using the peak signal-to-noise ratio as an evaluation index, image quality was improved by 0.29 dB for the male test data and by 0.24 dB for the female test data compared with the our previous serial model.
Otonona: a prototype for experiencing absolute pitch using multimodal augmented reality

The sense of sound, so called absolute and relative pitch, influences our perception and understanding of music. This research aims to extend the human sense of pitch in a multimodal approach using computers. In this paper, we focus on absolute pitch. Absolute pitch is an ability that some people are said to acquire by the age of six. Although degrees of this ability vary, people who have this ability are generally said to be able to recognize the note names even without being given a reference note. This is a potentially useful ability for understanding music, although it is difficult to acquire this ability through training as an adult. In this paper, we present the results of a subjective survey regarding with sound perception conducted on absolute pitch holders, the design and implementation of Otonona, a system for accessing the usefulness of absolute pitch using multimodal augmented reality, and future work.
Free-viewpoint Sports Video Generation Based on Dynamic NeRF Considering Time Series

Free-viewpoint video generation of sports games has been attracting attention. However, the conventional methods require a large number of cameras to generate a novel viewpoint, and it takes a lot of costs for introducing these technologies. To reduce these costs, in this paper, we present a NeRF-based free-viewpoint video generation method from real monocular sports videos. By considering time series of sports videos, our method can generate a realistic free-viewpoint video even from a single monocular video. Experimental results demonstrate the effectiveness of the proposed method.
Feature Extraction and Unsupervised Clustering of Histopathological Images of Pancreatic Cancer Using Information Maximization

In recent years, computer-aided diagnosis based on deep learning concepts has become an attractive research topic in medical imaging. Most of these works utilized supervised learning methods that required prior pathological knowledge. However, it is necessary to extract potential features in images, based on unsupervised learning in order to obtain new pathological findings. For this reason, we implemented unsupervised cluster analysis based on maximization of mutual information to classify pathological images of pancreatic cancer into discrete categories.
Campus School Affairs System Architecture and App for Course Push Broadcast Based on Smart Campus Applications

This paper proposes a campus school affairs system architecture and its dedicated campus mobile device app for the curriculum push broadcast. More comprehensive prevention on privacy and security is used to prevent mobile phone biometric identification. The proposed app can list the school bulletin board information and only need to operate the application on the mobile phone in a few seconds to get the school affairs information that is initially obtained. Furthermore, an emergency rescue function to contact the school security is also added so that the user can immediately notify the school for handling in the event of danger, saving the time of looking up the phone.
Summarization of Spoken Lectures Based on MMR Method and Important/Unimportant Sentence Classification Using BERT

In this study, we describe several summarization methods of lecture audio documents that use a distributed representation of sentences. First, we integrate a traditional summary system called maximal marginal relevance (MMR) with distributed sentence representation based on bidirectional encoder representations using Transformer (BERT). Second, we propose a summarization method based on the classification between important and unimportant sentences. Then, we integrate it with the BERT-based MMR method. Finally, we summarized spoken lectures in English and confirmed their effectiveness.
A Novel Subjective Perception Quality Evaluation Method of Video Based on EEG Signals

Traditional subjective QoE evaluation methods mainly rely on MOS ratings. However, the MOS ratings rely heavily on users’ subjective behavior feedback and are easily affected by emotional management, educational background and arbitrariness. The rating cannot show the accurate real-time quality of videos. With various wireless network communication technology, multimedia display technology and the continuous development of intelligent terminals, accurate assessment of the user experience is even more important. EEG signals, as electrophysiological signals that can reflect the thinking activities of human neural centers, can be quantified effectively without bringing into the evaluation of higher cognitive subjective bias, which is expected to more accurately measure users’ quality of experience. Therefore, aiming at the video quality degradation caused by the interaction between video stutter and video content, this paper proposes a method to measure video quality characterized by human EEG signal changes. The validity of QoE evaluation method based on EEG signal is proved by experimental data analysis.
Refinement of Gaze-based Image Caption for Image Retrieval

This paper presents refinement of trace-controlled image caption for image retrieval. With the development of cross-modal embedding techniques, recent studies show that it is effective to introduce visual trajectory information to assist the model in generating more accurate captions. However, there is a problem that the content of generated captions is not strictly corresponding to the trajectory, which results in the decentralization of semantic information. In this paper, we propose an image retrieval method combining image captioning model controlled by trajectory information called a connect caption and trace (CCT) model and refinement of the captions. By establishing a list of substantive words and comparing it with the generated captions output from the CCT model, the correspondence between captions and trace can be fastened and semantic information can be condensed. The effectiveness of our method is demonstrated by comparing it with a cross-modal embedding method in the experiments.
An Investigation of Code Modification Problem for Learning Server-side JavaScript Programming in Web Application System

Nowadays, JavaScript has become popular in web programming using Node.js, since it can be used on both client and server sides to make web pages interactive in a web application system. Previously, we have studied the code modification problem (CMP) for self-study of client-side programming for interactive web pages. A CMP instance requests to modify the given source code to satisfy the specified requirements. The correctness of any answer is checked through string matching with the correct one. In this paper, we investigate the CMP for self-study of server-side JavaScript programming. For its evaluations, we generated 10 instances on basic concepts in server-side programming and assigned them to 64 novice students in Japan and Indonesia. The solution results confirmed the validity of the proposal.
OmniVoxel: A Fast and Precise Reconstruction Method of Omnidirectional Neural Radiance Field

This paper proposes a method to reconstruct the neural radiance field with equirectangular omnidirectional images. Implicit neural scene representation with a radiance field can reconstruct the 3D shape of a scene continuously within a limited spatial area. However, training a fully implicit representation on commercial PC hardware requires a lot of time and computing resources (15~20 hours per scene). Therefore, we propose a method to accelerate this process significantly (20~40 minutes per scene). Instead of using a fully implicit representation of rays for radiance field reconstruction, we adopt feature voxels that contain density and color features in tensors. Considering omnidirectional equirectangular input and the camera layout, we use spherical voxelization for representation instead of cubic representation. Our voxelization method could balance the reconstruction quality of the inner scene and outer scene. In addition, we adopt the axis-aligned positional encoding method on the color features to increase the total image quality. Our method achieves satisfying empirical performance on synthetic datasets with random camera poses. Moreover, we test our method with real scenes which contain complex geometries and also achieve state-of-the-art performance. Our code and complete dataset will be released at the same time as the paper publication.
Evaluation of moving sound image rendering by VBAP and HOA with distance gain control

In this study, we investigated 3D moving sound image rendering methods using a spherical loudspeaker array surrounding the listener. We used vector base amplitude panning (VBAP), higher order ambisonics (HOA) panning, and near-field compensated HOA, which can control the sound image distance with a distance (radial) filter. The VBAP and HOA panning methods used gain control considering the distance between the listener and the image source. Subjective tests were conducted using a 42-channel geodesic dome-type spherical loudspeaker array. The experimental results showed that VBAP with distance gain control yielded the best results for both the smoothness and the localization of the moving sound image source, including the distance.
Proposal of an Immersive Tourism System in Vehicle for Seasonal Scenery

In this paper, we propose an immersive tourism system in vehicle for seasonal scenery. The proposed system projects a high-quality 360-degree video on the interior space of the vehicle by mixed reality (MR) technologies while synchronizing between the positions of the vehicle and the video. Passengers in the vehicle can experience the view of each season at the place to which a vehicle is traveling. At the early stage of the research, we investigated the video quality of the 360-degree video which was required for the immersive tourism system on a PC display and an HMD-type MR device. From an experiment, we found 4K video quality was required at least for the immersive tourism system and there was little difference between subjective evaluation values (MOS: mean opinion score) about the 4K video quality in the PC display and the MR device. We also developed an initial prototype system on a laptop PC in order to study feasibility of the proposed system. The prototype system realized continuous synchronization between the positions of the vehicle and the video to a certain degree although there was ample room for further improvement in the implementation.
A Tourist Spot Recommendation Method Based on Image Search and Sentiment Analysis

This paper proposes a method for supporting travel planning and sightseeing tours focusing on image search, to recommend tourist spots with high satisfaction based on sentiment analysis by computing image scores and sentiment scores. To compute image scores, we calculate both the visual and textual similarities between user-input photos and representative photos of tourist spots. To compute sentiment scores, we perform a sentiment analysis on the color information and the reviews of tourist spots to recommend tourist spots with high satisfaction. Finally, we evaluated and discussed our proposed method through the image search using the real sightseeing photo data.
Formulation of Ever Zooming-up Photomosaics

A photomosaic is one of artistic representations, which is produced by arranging hundreds of small photographs to form one target image. We formulate ever zooming-up mosaics (EZMs), which infinitely keeps transiting from a photomosaic into another. This limitless process is implemented with finding the best matching tile in the current mosaic and magnifying that tile for zooming. A straight-forward calculation requires an infeasible amount of comparisons for the typical mosaic size. We employ down-scaling of images and bit-wise operations for binary images to significantly reduce its computational costs. A preliminary experiment showed that bit-wise comparison between small image tiles was 30-75 times faster than ordinary loop-based pixel comparisons.
Valuation-Aware Federated Learning: An Auction-Based Approach for User Selection

Federated learning (FL) is a machine learning paradigm in which many users collaboratively train a model under the supervision of a central server (CS). Because the model performance is highly dependent on user quality, user selection becomes a critical issue in FL. In this paper, we develop a system model where the CS aims to select users with high computational power and valuation of the global model. In this regard, we propose an incentive mechanism to motivate users to reveal their computational power and model valuation. Then, we formulate a cost-minimization optimization problem of the CS and propose a polynomial-time dynamic programming algorithm to solve it. The proposed scheme effectively avoids the free-rider problem in which a user with little contribution can obtain the model by joining the FL process. Moreover, utilizing auction theory, our mechanism incentivizes users to report their computational power and model valuation truthfully. Finally, extensive theoretical analysis and numerical simulation validate the superiority of the proposed mechanism compared with two state-of-the-art user selection mechanisms.
High-speed image generation of transparent objects using an image conversion network

This paper proposes a method of generating images of transparent objects using pix2pix, which is a GAN-based image generation network, in order to shorten the rendering time of transparent objects in AR. We found that an image of a transparent object can be generated rapidly from a metallic object whose brightness varies greatly depending on the normal direction of the object.
A Mobile Application Presenting Hidden Architectural Heritage Sites in Augmented Reality

Urban development often endangers the preservation of architectural heritage sites, especially in metropolitan areas where architectural spaces are in high demand. As a result, many historical buildings that remain in use are temporarily covered with decoration to reduce the cost of their maintenance. In this study, we propose a context-aware mobile learning application designed to allow users to visually experience hidden historical architectural façades on-site and to present related knowledge. A successful field test showed that a working prototype was able to provide on-site viewing of a real hidden architectural façade. We expect the proposed approach to promote further awareness of the need for proper restoration of hidden architectural heritage sites.
Round Color 2D Symbol with Display Area and Secret Data Area for Business-to-Consumer Applications

The widely used QR code was developed for business-to-business applications for the manufacturing industry. Therefore, the QR code was not initially envisioned for business-to-consumer (B2C) applications, such as to refer consumers using smartphones to web addresses. In this regard, we examined the requirements for B2C applications. We developed a 2D symbol that is round and has color aesthetics, security functions, and a display area. We devised fan-shaped cells to yield a round symbol. It satisfies the requirements of cell distinctiveness and tessellation, which are the crucial for 2D symbols.