nils-herrmann/clean_scirepeval huggingface dataset:
Affective video segment retrieval for consumer generated videos based on correlation between emotions and emotional audio events

A novel affective video segment retrieval method based on the correlation between emotion and emotional audio events (EAEs) is presented. The proposed method focuses on retrieving three types of affective video segments, joy, sadness and excitement, by utilizing correlations between emotions and EAEs. The correlation between these emotions and EAEs is investigated by a subjective evaluation. The proposed method detects EAEs and rates each EAE in terms of emotion levels. The EAEs are detected by using the Generalized State-Space Model (GSSM) and low-level audio features. Experiments conducted on Consumer Generated Videos (CGVs) show that the proposed EAE detection outperforms conventional HMM and GMM based methods in terms of accuracy, the agreement rate of the retrieved affective video segments reaches 73.3%.
Video compression and enhancement sensors using column-parallel architecture

We propose two novel image sensors, compression sensor and motion adaptive sensor, which have computational elements on the sensor plane. They make use of column-parallel architecture. The compression sensor can significantly reduce the amount of pixel data in order to overcome the communication bottle neck for high pixel rate imaging by making use of very fast analog processing on the sensor plane. We also propose motion adaptive sensor for enhancing images and wider dynamic range. The proposed sensor is able to control the suitable storage time in each pixel which results in no motion-blur and no saturation. It is expected to have a wide dynamic range, high temporal resolution in the moving area and high SNR in the static area.
Pixel based random access image sensor array for real-time IBR

In the field of computer graphics, a new method called Image-Based Rendering (IBR) has been researched. With this method, 3D arbitrary-view images can be synthesized from array of input images. However, most of the current research is concentrated on static scenes because of the difficulty of handling a large amount of multiple video inputs. The authors adopt a random access image sensor for the camera array. In our system, each sensor outputs only the pixels that are externally selected, so that the total readout data from the camera array is reduced to that necessary and sufficient for the rendering. We designed and implemented the pixel based random access image sensor with 0.8 /spl mu/m CMOS technology. We implemented a camera array system with sixteen of the prototype devices. In this paper, we present our design of the random access CMOS image sensors and the construction of the camera array system. Evaluation of the prototype sensor and the results of the arbitrary-view synthesis are also described.
Personalization of food image analysis

This paper presents a method to classify food images by updating the model of Bayesian network incrementally. We have been investigating a “food log” system which makes use of image analysis, and it can automatically detect food images and estimate the food balance (using a simple nutrition model). It also enables users to easily modify the results of the analysis when they contain errors. So far, the system does not make use of the corrections made by the users to improve the performance of classification. In this paper, we propose to incrementally update the classifier based on Baysian network so that the results of analysis will be improved by using the user's corrections. With the incremental updating, the accuracy of food image detection is improved from 89% to 92%.
Smart image sensor for wide dynamic range by variable integration time

We propose a computational image sensor in which each pixel controls its integration time. The integration time of each pixel is selected from among several lengths of integration time and the integration time is shortened if the pixel is bright. Although the integration time of each pixel varies, the pixel intensity is adjusted on the sensor in real time. The dynamic range of the pixel value output from the proposed sensor is increased considerably. A prototype of 64/spl times/48 pixels has been fabricated. The proposed sensor has simple functions for detection of saturation to control the integration time and nonlinear image reconstruction. By using experiments, it has been verified that the sensor can achieve a wide dynamic range by adapting to light.
Residual Expansion Algorithm: Fast and Effective Optimization for Nonconvex Least Squares Problems

We propose the residual expansion (RE) algorithm: a global (or near-global) optimization method for nonconvex least squares problems. Unlike most existing nonconvex optimization techniques, the RE algorithm is not based on either stochastic or multi-point searches, therefore, it can achieve fast global optimization. Moreover, the RE algorithm is easy to implement and successful in high-dimensional optimization. The RE algorithm exhibits excellent empirical performance in terms of k-means clustering, point-set registration, optimized product quantization, and blind image deblurring.
Detecting Resized JPEG Images by Analyzing High Frequency Elements in DCT Coefficients

In this paper, we propose a method for detecting resized JPEG images. We defined 8 × 8 periodic blocks as JPEG blocks. JPEG block boundaries are detected by applying 8 × 8 block discrete cosine transform (DCT) to all the pixels of the input image and analyzing the high frequency coefficients in them. In order to quantitatively analyze the degree of forgery, we have developed two approaches such as truth-score and correlation-score methods. Experimental results using 375 original (untouched) JPEG images and 2,250 resized images recompressed with a variety of quality factors demonstrated that our proposed method can classify them with over 90% of accuracy. Our proposed method can detect not only conventional resizing but also state-of-the-art non-linear resizing such as seam carving.
Depth map up-sampling using cost-volume filtering

Depth maps captured by active sensors (e.g., ToF cameras and Kinect) typically suffer from poor spatial resolution, considerable amount of noise, and missing data. To overcome these problems, we propose a novel depth map up-sampling method which increases the resolution of the original depth map while effectively suppressing aliasing artifacts. Assuming that a registered high-resolution texture image is available, the cost-volume filtering framework is applied to this problem. Our experiments show that cost-volume filtering can generate the high-resolution depth map accurately and efficiently while preserving discontinuous object boundaries, which is often a challenge when various state-of-the-art algorithms are applied.
Uncalibrated Photometric Stereo by Stepwise Optimization Using Principal Components of Isotropic BRDFs

The uncalibrated photometric stereo problem for non-Lambertian surfaces is challenging because of the large number of unknowns and its ill-posed nature stemming from unknown reflectance functions. We propose a model that represents various isotropic reflectance functions by using the principal components of items in a dataset, and formulate the uncalibrated photometric stereo as a regression problem. We then solve it by stepwise optimization utilizing principal components in order of their eigenvalues. We have also developed two techniques that lead to convergence and highly accurate reconstruction, namely (1) a coarse-to-fine approach with normal grouping, and (2) a randomized multipoint search. Our experimental results with synthetic data showed that our method significantly outperformed previous methods. We also evaluated the algorithm in terms of real image data, where it gave good reconstruction results.
Focal plane compression and enhancement sensors

We propose two novel integrations of sensing and processing on the focal plane; they are a compression sensor and an enhancement sensor. The image compression sensor can significantly reduce the amount of pixel data in order to overcome the communication bottle neck of high pixel rate imaging between sensor and peripherals. The image enhancement sensor aims at reducing random noise and getting wide dynamic range. The proposed sensor controls the storage time in each pixel, which results in no motion-blur and no saturation. Thus it is expected to have wide dynamic range, high temporal resolution in the moving area and high SNR in the static area.
Motion estimation using multiple image sensors

A scheme of motion estimation using multiple image sensors is proposed, in which the multiple image sensors work at different integration intervals or different sampling instances. Because combined use of multiple image sensors virtually increases the temporal resolution while keeping signal to noise ratio high enough, the accuracy of the estimation can be improved. Both gradient-based and block-matching-based methods are formulated for the images obtained by the multiple image sensors. Tile experimental results show that the proposed scheme significantly improves the accuracy and reduces computational complexity.<<ETX>>
Multimedia ambiance communication

The photo-realistic 3-D image space of multimedia ambiance communication offers enhanced interpersonal communication. Formed by taking the laws of perspective and the characteristics of human visual perception into account, the space provides a natural environment that users can feel to be a part of. It is based on the concept of a three-layer structure, with long-, mid-, and short-range views. We constructed a testbed for multimedia ambiance communication consisting of a high-speed graphics computer and a curved screen onto which images are stereoscopically projected from the rear, and developed a three-camera system for capturing environment images. In addition, we developed the two-plane expression for processing backdrop views and highly efficient mid-range views. Finally, we constructed a photo-realistic 3-D image space using these image processing techniques.
Implementation of a 2D motion vector detection on image sensor focal plane

A vision chip for very fast detection of motion vector is presented. It consists of a pixel array and parallel block matching processors. Each pixel of the pixel array contains a photodetector, an edge detector and a 4-bit memory. The image data is firstly binarized by the edge detector and then the binary edge data is used in the block matching. The block matching is performed in local pixel parallel and global column parallel way. We can obtain a dense motion field in which a motion vector is assigned to each pixel by overlapping 2/spl times/2 target blocks. The prototype with 16/spl times/16 pixels and four block matching processors is designed and implemented. Test results obtained by the prototype are shown.
Sketch-Based Manga Retrieval Using Deep Features

Manga, Japanese comics, are globally popular, and the digital manga (e-manga) market is growing year by year. E-manga has a limitation in its search methodology: it is currently restricted to a keyword search of authors and titles. In this paper, we present an intuitive sketch-based manga retrieval method using deep features. We propose a framework to extract feature vectors from sketches and manga images by two differently trained CNNs: The two CNNs are trained on a large number of manga face images with and without screentone. The deep features are switched according to a query, that is, a sketch drawing or a crop of a manga image. We built an interactive retrieval system that has a browser interface. We evaluated its retrieval accuracy by using sketches and manga images. The proposed method significantly outperformed state-of-the art sketch-based manga retrieval using handcrafted features.
A degree-of-edit ranking for consumer generated video retrieval

We introduce degree-of-edit (DoE) ranking to focus on “how much a CGV is edited” as a ranking measure for consumer generated video (CGV) retrieval; a method to estimate DoE ranking is proposed. In the proposed method, the DoE score of a CGV is estimated by using low-level features such as the number of shot boundaries and time ratio of music. We evaluate the rank correlation between DoE ranking determined by subjects and by our method. To demonstrate its performance in a practical scenario, a user test is performed on over 22,000 CGVs in the context of CGV search. The obtained results show that our method significantly improves conventional CGV ranking results in terms of availabilities of interesting and high-quality CGVs.
Integrated macrocell/microcell (IMM) for traffic balancing in CDMA cellular system

This paper elucidates the problem of non-uniform traffic in CDMA cellular system. To alleviate the effect of non-uniform traffic, we propose a new CDMA cellular system that the same radio bandwidth is shared among the layers called integrated macrocell/microcell (IMM). The behavior of IMM is illustrated by traffic modeling. From the arriving traffic modeling, it shows that IMM provides a robustness to non-uniform traffic condition.
Internal noise-induced contrast enhancement of dark images

A contrast enhancement technique using scaling of internal noise of a dark image in discrete cosine transform (DCT) domain has been proposed in this paper. The mechanism of enhancement is attributed to noise-induced transition of DCT coefficients from a poor state to an enhanced state. This transition is effected by the internal noise present due to lack of sufficient illumination and can be modeled by a general bistable system exhibiting dynamic stochastic resonance. The proposed technique adopts a local adaptive processing and significantly enhances the image contrast and color information while ascertaining good perceptual quality. When compared with the existing enhancement techniques such as adaptive histogram equalization, gamma correction, single-scale retinex, multi-scale retinex, modified high-pass filtering, multi-contrast enhancement, multi-contrast enhancement with dynamic range compression, color enhancement by scaling, edge-preserving multi-scale decomposition and automatic controls of popular imaging tool, the proposed technique gives remarkable performance in terms of relative contrast enhancement, colorfulness and visual quality of enhanced image.
A layered method for determining manga text bubble reading order

Comic books of all cultures are an active research area as digitizing content for mobile and web is becoming more common. Past research on comics has largely concentrated on text extraction, panel segmentation and document analysis, while the utilisation of the extracted data has had less attention. In this paper we present a method to automatically determine the reading order of Japanese manga text bubbles using only text bubble position and image data. Our method classifies and orders page and text position information on three layers, which are hierarchically sorted to obtain the final ordering. The method is evaluated on a data set of 1769 manga pages with 14726 manually annotated text positions and correct ordering. Evaluation shows the method has over 95% transition accuracies and vastly outperforms a naive implementation.
A scheme for acquiring very high resolution images using multiple cameras

A signal processing-based scheme for acquiring high-resolution pictures with sufficiently high signal-to-noise ratio (SNR) by processing multiple charge-coupled device (CCD) images is presented. The method integrates multiple low-resolution images into a high-resolution image. Compared to a CCD imaging device with equivalent resolution, it may be less sensitive to shot noise which becomes more dominant as the pixel size of an imager is further reduced. Preliminary experiments using a camera model have shown clear improvements in the high frequencies and image details. A strategy to further improve resolution is also described.<<ETX>>
Comparative Analysis of Low-Level Visual Features for Affective Determination of Video Clips

Many algorithms and works have helped in the understanding and development of affective analysis of films. In spite of the progress made up to now, it is still not very precise how the low-level features of movies shape the resulting affective state of the viewer. In this work we evaluate different visual features and investigate how they impact the emotional evaluation under two paradigms: the dimensional approach (in terms of Pleasure, Arousal and Dominance) and the categorial approach. The analysis is conducted by using Dynamic Bayesian Networks (DBN) in the following topologies: a Hidden Markov Model network and Auto Regressive Hidden Markov Model network. Different combinations of feature vectors were used in order to check the performance of the proposed methods. The ground truth was created from an extensive user experiment, which was also used to create models that could map Pleasure, Arousal and Dominance values into affective categories.
Hyperlapse generation of omnidirectional videos by adaptive sampling based on 3D camera positions

Capturing a city with an omnidirectional camera can produce a continuous motion street view and provides the view of the moving person. However, watching such a video is not necessarily pleasant because the videos are often excessively lengthy. In this paper, we propose a method for shortening an omnidirectional video. Subsampling a video captured by a hand-held camera displays a significant amount of destabilization resulting from shaking. This prompted us to propose an adaptive subsampling scheme that selects optimal frames by minimizing our cost function based on 3D camera positions. This optimal selection suppresses only the translational camera instabilities. The rotational instabilities are initially ignored and later compensated for. This approach allowed us to successfully generate a subsampled and stable omnidirectional video. In addition, we propose an alternative measure of translational instability to evaluate our frame selection.
On sensor image compression for high pixel rate imaging

We propose a novel concept of an integration of compression and sensing in order to enhance performance of the image sensor. By integrating the compression function onto the sensor focal plane, the image signal that has to be readout from the sensor is significantly reduced. The integration consequently results in increasing the pixel rate of the sensor. Thus, the proposed sensors can be potentially applied to high pixel rate imaging such as high resolution image sensing and high frame rate image sensing. The compression scheme we make use of is conditional replenishment that detects and encodes moving areas. In this paper, we describe the design and implementation of two architectures for sensor compression; one is a pixel parallel approach and the other is a column parallel approach. We prototyped a VLSI chip of the proposed sensor based on the pixel parallel architecture and we describe the details of the results of the experiments obtained by the prototype chip.
Reconstructing arbitrarily focused images from two differently focused images using linear filters

We present a novel filtering method for reconstructing an all-in-focus image or an arbitrarily focused image from two images that are focused differently. The method can arbitrarily manipulate the degree of blur of the objects using linear filters without segmentation. The filters are uniquely determined from a linear imaging model in the Fourier domain. An effective and accurate blur estimation method is developed. The simulation results show that the accuracy and computational time of the proposed method are improved compared with the previous iterative method and that the effects of blur estimation error on the quality of the reconstructed image are very small. The method performs well for real images acquired without visible artifacts.
Similar floor plan retrieval featuring multi-task learning of layout type classification and room presence prediction

In this paper, a new framework for real estate property searches is presented in which a floor plan image is used as a query. In similar property searches, appearance-based similar image retrieval does not work well because similar properties have totally different floor plan images. Therefore, a multitask learning method using deep neural networks to solve this problem is presented. Convolutional Neural Networks (CNNs) are trained to solve the two tasks: layout type classification and room presence classification. The feature vectors obtained from the CNNs are then applied to the retrieval task. Experiments using 22,140 floor plan images in Tokyo, Japan were conducted, and the proposed method achieved the best performance (15.7% with precision@5) compared to other possible approaches.
Automatic preview video generation for mesh sequences

We present a novel method that automatically generates a preview video of a mesh sequence. To make the preview appealing to users, the important features of the mesh model should be captured in the preview video, while preserving the constraint that the transitions of the camera are as smooth as possible. Our approach models the important features by defining a surface saliency and by measuring the appearance of the mesh sequence. The task of generating the preview video is then formulated as a shortest-path problem and we find an optimal camera path by using Dijkstra's algorithm.
3D Video Compression Based on Extended Block Matching Algorithm

Three dimensional (3D) video is attracting a lot of attention as a new multimedia representation method. 3D video is a sequence of 3D models (frames) that consist of varying vertices and connectivity. In conventional 2D video compression algorithms, motion compensation (MC) using block matching algorithm is frequently employed to reduce redundancy between consecutive frames. However, there is no such technology for 3D video so far. Therefore, in this paper, we have developed an extended block matching algorithm (EMBA) to reduce temporal redundancy of geometry information of 3D video by extending the idea of 2D block matching to 3D space. In our EBMA, a cubic block is used as a matching unit and, MC is achieved efficiently by matching the mean normal vectors of the sub-blocks, which turned out to be sub-optimal by our experiments. The residual information is further transformed by discrete cosine transform (DCT) and then encoded. The extracted motion vectors are also entropy encoded. As a result of our experiments, compression ratio ranging from 10% to 18% of the original 3D video data has been achieved.
Depth map inpainting and super-resolution based on internal statistics of geometry and appearance

Depth maps captured by multiple sensors often suffer from poor resolution and missing pixels caused by low reflectivity and occlusions in the scene. To address these problems, we propose a combined framework of patch-based inpainting and super-resolution. Unlike previous works, which relied solely on depth information, we explicitly take advantage of the internal statistics of a depth map and a registered highresolution texture image that capture the same scene. We account these statistics to locate non-local patches for hole filling and constrain the sparse coding-based super-resolution problem. Extensive evaluations are performed and show the state-of-the-art performance when using real-world datasets.
Digital watermarking using inter-block correlation

In this paper, we propose a new watermark technique which utilizes the inter-block correlation of DCT coefficients. The amount of modification, that is the strength of embedded watermark, depends on the local feature of an image such that the watermark can be strongly embedded in the region where signal distortion is perceptually less visible. This feature also makes difficult for pirate to predict the position in which the watermark signal is embedded, since the amount of modification varies in the image. Embedded watermark can be detected without the parameters used in embedding process nor the original image. The experimental results show that this technique is robust to JPEG compression and Gaussian noise.