Reviewerly/exHarmony huggingface dataset:
Improved probability modeling for lossless image coding using example search and adaptive prediction

We previously proposed a novel lossless image coding method that utilizes example search and adaptive prediction within a framework of probability model optimization. In this paper, the definition of the probability model as well as its optimization procedure are modified to reduce the encoding complexity. In addition, affine predictors used in the adaptive prediction are refined for accurate probability modeling. Simulation results indicate that our modification contributes not only to encoding time reduction, but also to coding efficiency improvement for all of the tested images.
Depth scene flow estimation based on variational method using thin-plate spline regularization

Scene flow is a three-dimensional (3D) vector field with depth directional motion and optical flow and it can be applied to inter prediction in 3D video coding. The conventional method regularizes the scene flow so that it locally approaches a constant function, so there is the problem that it is difficult to handle spatial changes and motion boundaries. Regularizations called thin-plate spline or deformable model have been introduced in variational optical flow estimation because they find the solution locally as a linear function and may alleviate the problem. However, because the partial differential equation derived by thin-plate spline regularization includes the fourth-order partial differential, it is not easy to derive an analytical solution or solve it by a numerically stable iterative method in terms of numerical analysis. Previous researches have proposed a numerically stable iterative method that does not include thin plate spline regularization for scene flow estimation. Therefore, while making use of the framework of the previous researches, I derive a partial differential equation using thin plate spline regularization for scene flow estimation.
Fuzzy Adaptive Teaching Learning-Based Optimization for Solving Unconstrained Numerical Optimization Problems

Teaching learning-based optimization is one of the widely accepted metaheuristic algorithms inspired by teaching and learning within classrooms. It has successfully addressed several real-world optimization problems, but it may still be trapped in local optima and may suffer from the problem of premature convergence in the case of solving some challenging optimization problems. To overcome these drawbacks and to achieve an appropriate percentage of exploitation and exploration, this study presents a new modified teaching learning-based optimization algorithm called the fuzzy adaptive teaching learning-based optimization algorithm. The proposed fuzzy adaptive teaching learning-based optimization algorithm uses three measures from the search space, namely, quality measure, diversification measure, and intensification measure. As the 50-50 probabilities for exploitation and exploration in the basic teaching learning-based optimization algorithm may be counterproductive, the Mamdani-type fuzzy inference system of the new algorithm takes these measures as a crisp inputs and generates selection as crisp output to choose either exploitation or exploration based on the current search requirement. This fuzzy-based adaptive selection helps to adequately balance global search or exploration and local search or exploitation operations during the search process as these operations are intrinsically dynamic. The performance of the fuzzy adaptive teaching learning-based optimization is evaluated against other metaheuristic algorithms including basic teaching learning-based optimization on 23 unconstrained global test functions. Moreover, adaptive teaching learning-based optimization is used to search for near-optimal values for the four parameters of the COCOMO II model, which are then tested for validity on a software project of NASA. Analysis and comparison of the obtained results indicate the efficiency and competitiveness of the proposed algorithm in addressing unconstrained continuous optimization tasks.
Kernel estimation for super-resolution with flow-based kernel prior

Single-Image Super-Resolution methods typically assume that a low-resolution image is degraded from a high-resolution one through “bicubic” kernel convolution followed by downscaling. However, this induces a domain gap between training image datasets and the real scenario’s test images, which are down-sampled from the images that underwent convolution with arbitrary unknown kernels. Hence, correct kernel estimation for a given real-world image is necessary for its better super-resolution. One of the kernel estimation methods, KernelGAN locates the input image in the same domain of high-resolution image for accurate estimation. However, using only a low-resolution image cannot fully utilize the high-frequency information in the original image. To increase the estimation accuracy, we adopt a superresolved image for kernel estimation. Also, we use a flow-based kernel prior to getting a reasonable super-resolved image and stabilize the whole estimation process.
An immersive self-training system of receive motion for volleyball beginners

In this article, we propose a VR-based immersive self-training system for volleyball beginner’s receiving, by which a virtual training environment is given to a practitioner with a head-mounted-display (HMD). Not wearing motion capture markers and haptic devices, a practitioner has free hands to receiving a virtual ball, so that the proposed system is expected him/her to be able to play receiving motion as the same as in the real world, in order to enhance skill upgrading effectively.
Multi-level feature aggregation network for high dynamic range imaging

Modern digital cameras typically cannot capture the whole range of illumination, due to the limited sensing capability of sensor devices. High dynamic range (HDR) imaging aims to generate images with a larger range of illumination by merging multiple low-dynamic range (LDR) images with different exposure times. However, when the images are captured in dynamic scenes, existing methods unavoidably produce undesirable artifacts and distorted content. In this paper, we propose a multi-level feature aggregation network, based on the Laplacian pyramid, to address this issue for HDR imaging. The proposed method progressively aggregates non-overlapping frequency sub-bands at different pyramid levels, and generates the corresponding HDR image from coarser to finer scales. Experiment results show that our proposed method can significantly outperform other competitive HDR methods, thereby producing HDR images with high visual quality.
Posture estimation for the visually impaired people using human skeleton with a white cane

Visually impaired people avoid dangers by using a white cane. Because they use it as an extension of their arm, we consider it a part of their body. Our goal is to recognize visually impaired people using a white cane in images and how they use it. We propose a method to estimate the posture of a person with a white cane by extending an existing posture estimation model: OpenPose. In our method, we incorporate a white cane as a part of the human skeleton model. We constructed a database of images of visually impaired people with a white cane to train the network for the extended human skeleton model. We develop a method to determine the left or right hand that holds the white cane in the training images because it is necessary to train right-handed and left-handed users separately. We can analyze the motion of the white cane by the result of posture estimation. We focus on the angle of the white cane and analyze its swing frequency. Throughout our experiments, we confirmed that our preliminary system successfully estimated the human posture with the white cane and the swing frequency of the white cane.
Comparison of real-time CNN-based methods for finger-level hand segmentation

Hand segmentation is usually considered a pixel-wise binary classification problem, where the foreground hand is meant to be recognized in an input image. However, we envision that finger-level hand segmentation is more useful for applications like hand gesture and sign language recognition. Therefore, in this paper, we compare five state-of-the-art (SOTA) real-time semantic segmentation methods for the task of finger-level hand segmentation. To do that, we introduce two subsets consisted of 1,000 images manually annotated pixel-wise selected from new proposed datasets of hand gesture and world-level sign language recognition. With these subsets, we evaluate the accuracy of the recent SOTA methods of DABNet, FastSCNN, FC-HardNet, FASSDNet, and DDRNet. Since each subset has relatively few images (500), we introduce a simple yet effective loss function to train with synthetic data that includes the same annotations. Finally, we present a real-time performance evaluation of the five algorithms on the NVIDIA Jetson family of GPU-powered embedded systems, including Jetson Xavier NX, Jetson TX2, and Jetson Nano.
Generation of dynamic images for fake-face detection

Recently, a video of famous politicians and superstars giving certain speeches surfaced online causing severe political and commercial problems. The video, although it seemed authentic, was fake. Therefore, in this study, using 37,936 videos sampled from the deep fake detection challenge (DFDC), we developed an efficient and highly accurate deepfake detection system using EfficientNet with dynamic images. Dynamic image transforms video sequences into one frame instance by conserving spatiotemporal information. The experimental results and comparative analysis indicate that EfficientNet with dynamic image exhibits higher performance than EfficientNet. We also found that dynamic images generated by 20 frames have a higher fake-face detection accuracy than simple images.
Proxy system with JPEG bitstream-based file-size preserving encryption for cloud photo streams

In this paper, we propose a proxy system with JPEG bitstream-based file-size preserving encryption to securely store compressed images in cloud environments. The proposed system, which is settled between client’s device and the Internet, allows us not only to have exact the same file size as that of original JPEG streams but also to maintain a predetermined image format. In an experiment, the proposed system is verified to be effective in two cloud photo steams: Google Photo and iCloud Photo.
A method of feature map reordering for machine vision based on channel correlation

As the need for a video coding technology for a machine that performs intelligent analysis such as object detection, segmentation, and tracking on massive video data has emerged, MPEG is developing a standard called video coding for machines (VCM). VCM is a standard technology for compression of image/video or its features for performing vision tasks of intelligent machines. In this paper, we propose methods that convert multichannel features extracted from an analysis network of input images into a reordered feature map sequence for enhanced compression using VVC. The proposed methods exploit the correlation between channel feature maps using their mean values and sum of absolute difference (SAD) between feature maps in the reordering. Although the proposed methods do not reach the anchor performance of VCM, it shows better coding performance than compressing the feature without channel reordering.
On the instability of unsupervised domain adaptation with ADDA

In this paper we report the instability of Adversarial Discriminative Domain Adaptation (ADDA), an unsupervised domain adaptation. The accuracy of ADDA is not stable, and we show that the instability comes from the initialization of CNN for the target domain, not from the pre-training with the source domain.
Region-aware point cloud sampling for hand gesture recognition

In this paper, we propose a region-aware point cloud sampling method that adaptively divides regions containing important information for hand gesture analysis. Experimental results prove that the proposed method contributes to improving hand gesture recognition performance.
Multi-level unsupervised domain adaption for privacy-protected in-bed pose estimation

In-bed pose estimation is of great value in current health-monitoring systems. In this paper, we solve a crossdomain pose estimation problem, in which a fully annotated uncovered training set is used for pose estimation learning, and a large-scale unlabelled data set of covered images is employed for unsupervised domain adaptation. To tackle this challenging problem, we propose a multi-level domain adaptation framework, which learns a generalizable pose estimation network based three levels of adaptation. We evaluate the proposed framework on a public in-bed pose estimation benchmark. The results demonstrate that our proposed framework can effectively generalize the learned knowledge from the uncovered source domain to the covered target domain for privacy-protected in-bed pose estimation.
An Adaptive UI Based on User-Satisfaction Prediction in Mixed Reality

As people begin to notice mixed reality, various studies on user satisfaction in mixed reality (MR) have been conducted. User interface (UI) is one of the representative factors that affect interaction satisfaction in MR. In conventional platforms such as mobile devices and personal computers, various studies have been conducted on providing adaptive UI, and recently, such studies have also been conducted in MR environments. However, there have been few studies on providing an adaptive UI based on interaction satisfaction. Therefore, in this paper, we propose a method based on interaction-satisfaction prediction to provide an adaptive UI in MR. The proposed method predicts interaction satisfaction based on interaction information (gaze, hand, head, object) and provides an adaptive UI based on predicted interaction satisfaction. To develop the proposed method, an experiment to measure data was performed, and a user-satisfaction-prediction model was developed based on the data collected through the experiment. Next, to evaluate the proposed method, an adaptive UI providing an application using the developed user-satisfaction-prediction model was implemented. From the experimental results using the implemented application, it was confirmed that the proposed method could improve user satisfaction compared to the conventional method.
Deep learning-based feature compression for video coding for machine

We previously trained the compression network via optimization of bit-rate and distortion (feature domain MSE) [1]. In this paper, we propose feature map compression method for video coding for machine (VCM) based on deep learning-based compression network that joint training for optimizing both compressed bit rate and machine vision task performance. We use bmshij2018-hyperporior model in the CompressAI [2] as the compression network, and compress the feature map which is the output of stem layer in the Faster R-CNN X101-FPN network of Detectron2 [3]. We evaluated the proposed method by evaluation framework for MPEG VCM. The proposed method shows the better results than VVC of MPEG VCM anchor.
Dialogue-Filling: Response Generation Control in Retrieval-Generation Dialogue System

Generation-base dialogue system tends to produce generic response sentences. In order to improve the diversity of response sentences by the generation-base dialogue system, the response text retrieved by the retrieval-base model can be input to the generation-base model as reference response text, so that the generation-base model can generate highly diverse response sentences. However, the prior works show that the generation-base dialogue system often ignores the reference response text, resulting in the response sentences that is unrelated to the reference response text. In this work, we propose the Dialogue-Filling method, which can utilize 100% of the reference response text by masking the response sentences with a text-filling technique. We built variants of Dialogue-Filling method with DialoGPT model. Experiments on the DailyDialog Dataset demonstrate that our Dialogue-Filling method outperforms the baseline method on the dialogue generation task.
Descriptor-based video coding for machine for multi-task

The coding objective of image and video that are targeted for machine consumption may differ from that for human consumption. For example, machine may only use a part of image or video requested or required by an application whereas human consumption requires whole captured area of image and video. In addition, machine may require grayscale or certain light spectrum, whereas human consumption requires full visible light spectrum. To identify an object of interest, a neural network based image or video analysis task may be performed and the output of a task is an identified feature (latent) and an associated descriptor (inference). Depending on the usage, multiple tasks can be performed in parallel or in series, and as a number of identified feature increases, the chance of feature area overlap increases as well. We propose a pipeline of descriptor based video coding for machine for multi-task. The proposed method is expected to increase coding efficiency when multiple tasks are performed, by minimizing redundant encoding of overlapped area of objects of interest and to increase utilization and re-utilization of features by transmitting inference separately.
Anderson relaxation test for intrinsic dimension selection in model-based clustering

Parsimonious finite mixture models often require the a priori selection of desired model dimensionality. For example, projection-based parsimonious models demand the dimension of the subspace for projection. Other models ask for their own structural restrictions on parameters. The subspace clustering framework is a projection-based parsimonious model for various finite mixtures, including the Gaussian variant. The existing dimension selection methods for subspace clustering are ad-hoc or potentially computationally prohibitive, creating a need for a principled, yet computationally lightweight, approach. In light of this problem, a hypothesis test-based intrinsic dimension estimation method called the Anderson Relaxation Test (ART) is introduced, and its performance is examined in both simulated and real data settings.
Pruning Stochastic Game Trees Using Neural Networks for Reduced Action Space Approximation

Monte Carlo Tree Search has proved to be very efficient in the broad domain of Game AI, though it suffers from high dimensionality in cases of large branching factors. Several pruning techniques have been proposed to tackle this problem, most of which require explicit domain knowledge. In this study, an approach using neural networks to determine the number of actions to be pruned, depending on the iterations run and the total number of possible actions, is proposed. Multi-armed bandit simulations with the UCB1 formula are employed to generate suitable datasets for the networks’ training and a specifically designed process is followed to select the best combination of the number of iterations and actions for pruning. Two pruning Monte Carlo Tree Search variants are investigated, based on different actions’ expected rewards’ distributions, and they are evaluated in the collectible card game Hearthstone. The proposed technique improves the performance of the Monte Carlo Tree Search algorithm in different setups of computational limitations regarding the available number of tree search iterations and is significantly boosted when combined with supervised learning trained-state value predicting models.
Input Utterance Complementation Method by Anaphora Resolution for Spontaneous Utterances on Spoken Dialog Systems

In this study, we propose a method for generating response utterances which take into account contexts and topics of the dialog by complementing omitted words such as subjects in the input utterances of dialog systems. In order to complement omitted words in the input utterances, an automatic anaphora resolution based on the centering theory is performed. To achieve highly accurate anaphora resolution, we also performed spoken-to-written style conversion based on sequence-to-sequence model using LSTM as a preprocessing. The results of evaluation experiments using NUCC, the Nagoya University Conversation Corpus showed that our proposed complementation method works robustly against errors in spoken-to-written style conversion.
A high-accuracy circle detection method using a multi-angle rotating equilateral triangle

Nowadays, visual inspection is widely used in manufacturing industries such as electronic industries, and the most common shape detection is circle detection. Although, there were many methods proposed for circle detection, there is a room to improve the detection accuracy. This paper then proposes a high-accuracy circle detection method using a multiangle rotating equilateral triangle. The main idea is to fit the equilateral triangle into the circle with various rotation angles to select the most accurate circle center. Triangles with various rotation angles are fitted into the circle by iteratively adjusting the ratios of the triangle bases to another sides. The triangles which cannot be adjusted to be equilateral are then discarded, because those isosceles triangles, which occur when a detected triangle corner (s) is not on the circumference but on noises inside the circle, are the cause of inaccurately detected circle centers. The experimental results show that the proposed method is superior to the conventional methods in terms of accuracy and processing time.
Vehicle re-identification system for road side unit application

Vehicle re-identification is one of the complex smart traffic issues, due to the large intra-class variation and high interclass similarity, it is hard to be solved through traditional hand-crafted features. In this paper, we proposed a vehicle re-identification system based on deep learning techniques, which is able to re-identify the vehicles through deep features under acceptable operation time on Nvidia Jetson TX2. We have collected multiple sequences captured from real-world road side units (RSU) for system evaluation experiments, and the results indicate that it is highly possible to be adopted for real world traffic applications.
Robust Slot Filling Modeling for Incomplete Annotations using Segmentation-Based Formulation

In language understanding for dialog systems, slot filling is a fundamental task usually formulated as a sequence labeling problem and solved using discriminative models such as conditional random fields and neural networks. One of the weak points of the discriminative approach is in the robustness against incomplete annotations, which are often generated in practice when we attempt to build large-scale training data. For making the slot filling algorithm more robust against the incompleteness of annotation, this paper leverages an overlooked property of slot filling tasks: Non-slot parts of utterance follow a specific pattern depending on the user’s intent. To reflect this idea, we propose a nonparametric Bayesian model that induces the grammatical role of the non-slot parts using a segmentation-based formulation of slot filling tasks. The proposed method can naturally deal with training data that includes incomplete annotations as a partially supervised grammar induction problem. The experimental result demonstrates that the proposed method estimates the slot information more accurately in a situation that the training data includes incomplete annotations in comparison to the BiLSTM-CRF and HMM.We also show that the proposed model has an advantage in the interpretability of the result of training and prediction by visualizing the parameters and the estimated labeled segmentations with a state transition diagram.