nils-herrmann/clean_scirepeval huggingface dataset:
PicWall: Photo collage on-the-fly

Photo collage, which constructs a compact and visually appealing representation from a collection of input images, provides the best convenient and impressive user experience. Previous approaches for automatic collage generation are always analogized as optimization problems, in which the researchers are trying to find the best balance between maximizing the visibility of photos' salient areas as well as compactly arrange the collage canvas layout. However, automatic saliency detection can sometimes be harmful since we cannot guarantee all of the user's interest areas are well-kept. On the other hand, the rapid development of mobile technology also calls for a robust solution of fast collage generation without any computation-expensive processes such as saliency detection and graph-cut. The PicWall approach proposed in this paper offers real-time collage generation. Given the expected canvas sizes, it tightly packs the input images while keeping their aspect ratios and orientations unchanged. Experiments show that it costs less than 0.5ms for a 100-photo collage generation. Besides, various PicWall-based applications are also demonstrated.
3D video segmentation using point distance histograms

Similar to 2D video segmentation, 3D video segmentation is to divide the 3D video in temporal domain into a set of meaningful and manageable segments (shots) that are used as basic elements for indexing. This paper proposes a temporal segmentation method for 3D video for the first time as far as we know. Point distance histograms are used considering the tradeoff between computational cost and effectiveness. In order to reflect the real motion of 3D object, three fixed points are selected, which can avoid what we call "the same sphere problem." And simulation results on total 285 frames, which are composed of four different sequences, show our method is very effective.
Tracking Persons using Particle Filter Fusing Visual and Wi-Fi Localizations for Widely Distributed Camera

This paper describes an object tracking scheme employs sensor fusion approach which is composed of visual information and location information estimated from Wi-Fi signals. Location information is calculated by a set of received signal strength values of beacon packets from Wi-Fi access points (APs) around the targets. Different from the conventional approaches which use another kind of sensors, our approach can cover wider areas both indoor and outdoor with lower cost because of characteristics of Wi-Fi signals. Particle filter is applied to combine these two different kinds of sensory input to track the target continuously. Wi-Fi observation model is involved in a conventional visual particle filtering scheme in order to evaluate importance weights of each particle. By using multiple modality, robust tracking performance is achieved even if reliability of one sensory input declines. In this paper, we present experimental results applied to outdoor surveillance camera environment.
Noise attenuation performance of mura apertures in photographic cameras

Coded apertures provide enhanced noise characteristics to lens-less imaging systems. Similar characteristics have been suggested for traditional cameras. Four experiments were conducted to investigate the amount of noise reduction provided by coded apertures when used in photographic cameras. They focus on the effects of coding mask sizes, positioning, mosaicing and rotation. The signal to noise ratio of photos taken with and without coding masks was evaluated. The mask pattern utilized was the modified uniformly redundant array (MURA). The experiments showed evidence of noise reduction and best results were obtained for masks with higher element count, mosaiced and limited to the sensor size.
Clustering meal images in a web-based dietary management system

We investigate the possibility of summarizing meal image collections in an image-based dietary assessment system. A segmentation algorithm detects food regions in each meal image. Pairwise matching of images based on color and texture information of these regions forms a similarity matrix of the image collection. We cluster the nodes of the graph constructed using this matrix, to identify natural groupings of meal images according to their content. Representative images from these clusters form summaries of the large image collection. We conduct a user study to evaluate the effectiveness of the proposed algorithms in summarizing meal image collections, and report the results.
Motion Segmentation of 3D Video using Modified Shape Distribution

In this paper, temporal segmentation of 3D video based on motion analysis is presented. 3D video is a sequence of 3D models made for a real-world dynamic object. A modified shape distribution algorithm is proposed to realize stable shape feature representation. In our approach, representative points are generated by clustering vertices based on their spatial distribution instead of randomly sampling vertices as in the original shape distribution algorithm. Motion segmentation is conducted analyzing local minima in degree of motion calculated in the feature vector space. The segmentation algorithm developed in this paper does not require any predefined threshold values but rely on relative relationships among local minima and local maxima of the motion. Therefore, robust segmentation has been achieved. The experiments using 3D video of traditional dances yielded encouraging results with the precision and recall rates of 93% and 88%, respectively, on average
Fast and Efficient Normal MAP Compression Based on Vector Quantization

Normal maps play an important role in realistic 3D image rendering to express pseudo roughness of the surface with small amount of polygon data. In this paper, a fast and efficient normal map compression algorithm is proposed based on vector quantization and entropy coding. Using the strong correlation among x, y, and z components of normal maps owing to the unity condition, compression ratio has been made much better than conventional approaches. In addition, the encoding time has been made reasonable by considering the distribution of the data and employing inner product in nearest-neighbor search instead of Euclidian distance taking advantage of the unity condition of the training data
Generation of representative meal names for food recording data by using web search results

A meal is often called by different names depending on cultures or individuals, and a significantly large number of meal names exist in food recording data from the general public. To analyze the statistics of meals in food recording data, we need to normalize meal names; in other words, make representative names for food recording data. In this paper, we present our approach to obtaining representative names by using both food recording data and web search results. Firstly, we obtained snippets of documents from the web search results for a target meal name. Next, we performed a morphological analysis and recognized recipe-named entities (recipe NEs). Finally, we obtained the most frequent words under the condition that they were included in a candidate list of representative names. We created a dataset and evaluated our proposed method. As a result, 47.96% of Top 1 candidates and 77.32% of Top 5 candidates for generated representative names coincided with those of the ground truth (GT).
Data-Driven Geometric Face Image Smilization Featuring Moving Least Square Based Deformation

With the prosperous development of the Internet, virtual identity is becoming more important to people in social life. As one key factor, attractiveness, is a significant part of virtual identity, where there is still room for different strategies to improve attractiveness. In this study, we focus on digital face image and make it a little bit smiling as a common way to improve attractiveness. The workflow includes face detection, face alignment and image deformation. By only deforming geometrically, our method reaches the smile rate at 53.96% while natural smile image has 89.29% smile rate, based on crowd sourcing experimental results on our test set.
Prediction of users' facial attractiveness on an online dating website

Online dating websites are popular platforms for adults to search for their life partners. Because on online dating websites, a user's profile image is an important factor determining other's impressions, we focus on profile images and analyze users' visual attractiveness in this study. Facial attractiveness is strongly related to our perception of aesthetics and therefore we believe our investigation can somewhat contribute to artwork analysis. We use pre-trained convolutional neural networks (CNN) to extract visual features and propose a new method to rank users' attractiveness from their online dating interactions. For both genders, we predict users' facial attractiveness by supervised machine learning. Our experimental results show that deep representations of profile images are powerful to capture facial attributes' differences and perform well in predicting users' attractiveness. The correlation coefficient of 0.462 for male users and the correlation coefficient of 0.387 for females users is obtained for regression. The accuracy of 75% for females and the accuracy of 78.8% for males is obtained for 2-level classification.
Food Balance Estimation by Using Personal Dietary Tendencies in a Multimedia Food Log

We have investigated the “FoodLog” multimedia food-recording tool, whereby users upload photographs of their meals and a food diary is constructed using image-processing functions such as food-image detection and food-balance estimation. In this paper, following a brief introduction to FoodLog, we propose a Bayesian framework that makes use of personal dietary tendencies to improve both food-image detection and food-balance estimation. The Bayesian framework facilitates incremental learning. It incorporates three personal dietary tendencies that influence food analysis: likelihood, prior distribution, and mealtime category. In the evaluation of the proposed method using images uploaded to FoodLog, both food-image detection and food-balance estimation are improved. In particular, in the food-balance estimation, the mean absolute error is significantly reduced from 0.69 servings to 0.28 servings on average for two persons using more than 200 personal images, and 0.59 servings to 0.48 servings on average for four persons using 100 personal images. Among the works analyzing food images, this is the first to make use of statistical personal bias to improve the performance of the analysis.
DrawFromDrawings: 2D Drawing Assistance via Stroke Interpolation with a Sketch Database

We present DrawFromDrawings, an interactive drawing system that provides users with visual feedback for assistance in 2D drawing using a database of sketch images. Following the traditional imitation and emulation training from art education, DrawFromDrawings enables users to retrieve and refer to a sketch image stored in a database and provides them with various novel strokes as suggestive or deformation feedback. Given regions of interest (ROIs) in the user and reference sketches, DrawFromDrawings detects as-long-as-possible (ALAP) stroke segments and the correspondences between user and reference sketches that are the key to computing seamless interpolations. The stroke-level interpolations are parametrized with the user strokes, the reference strokes, and new strokes created by warping the reference strokes based on the user and reference ROI shapes, and the user study indicated that the interpolation could produce various reasonable strokes varying in shapes and complexity. DrawFromDrawings allows users to either replace their strokes with interpolated strokes (deformation feedback) or overlays interpolated strokes onto their strokes (suggestive feedback). The other user studies on the feedback modes indicated that the suggestive feedback enabled drawers to develop and render their ideas using their own stroke style, whereas the deformation feedback enabled them to finish the sketch composition quickly.
Spatially variant flexible sampling control integrated on an image sensor

We propose a new sampling control system integrated on an image sensor. Contrary to the conventional random access pixels, the proposed sensor is able to read out spatially variant pixels at high speed, without inputting pixel address for each access. The sampling positions can be changed dynamically by rewriting the sampling position memory. Since the proposed sensor has an array memory that keeps the pixel position to be sampled. The sampling position can be dynamically changed by rewriting the memory array. It can achieve any spatially varying sampling patterns. We have made a first prototype and show results obtained by the prototype.
Digital watermarking using inter-block correlation: extension to JPEG coded domain

Digital watermarking schemes have been discussed to solve the problem associated with copyright enforcement. As a digital watermarking, we proposed a method using inter-block correlation of DCT coefficients. It has the features that the embedded watermark can be extracted without the original image nor the parameters used in embedding process and that the amount of modification, the strength of embedded watermark, depends on the local feature of an image. This feature makes it difficult for pirate to predict the position in which the watermark signal is embedded. We propose a method which can embed/extract watermark with high speed by utilizing this watermarking method for JPEG file format.
Prediction of the time to pregnancy by kernel density estimation on lifestyle questionnaire

Current pregnancy measures can be classified into three types: natural pregnancy, artificial insemination, and in vitro fertilization. For those who are trying to get pregnanct, however, it is difficult to judge when they should switch from natural pregnancy to other methods because the factors that contribute to pregnancy are usually not clear. This paper presents a personalized method to estimate the time to pregnancy (TTP) for a natural conception by using kernel density estimation and k-nearest neighbor. Using this method, we can provide suggestions and a TTP estimation to guide decisions on which methods and actioins are suitable for those willing to conceive. In addition, we developed a user-friendly GUI system that can show the probability of the TTP and provide feedback through tips without requiring user expertise.
An image processing algorithm for a super high definition imaging scheme with multiple different-aperture cameras

Towards the development of a SHD (super high definition) image acquisition system, previously the authors developed the image-processing based approach with multiple cameras. Originally, in this approach, they used multiple cameras with the same pixel aperture, but in this case there needs to be severe limitations both in the arrangement of multiple cameras and in the configuration of the scene in order to guarantee the spatial uniformity of the resultant resolution. To overcome this difficulty completely, the authors have also previously presented the utilization of multiple cameras with different pixel apertures. The present paper develops a new, alternately iterative image processing algorithm available in the different aperture case. Experimental simulations clearly show that the alternately interactive algorithm behaves satisfactorily.<<ETX>>
Boosting VLAD with double assignment using deep features for action recognition in videos

The encoding method is an important factor for an action recognition pipeline. One of the key points for the encoding method is the assignment step. A very widely used super-vector encoding method is the vector of locally aggregated descriptors (VLAD), with very competitive results in many tasks. However, it considers only hard assignment and the criteria for the assignment is performed only from the features side, by looking for which visual word the features are voting. In this work we propose to encode deep features for videos using a double assignment VLAD (DA-VLAD). In addition to the traditional assignment for VLAD we perform a second assignment by taking into account the perspective from the codebook side: which are the nearest features to a visual word and not only which is the nearest centroid for the features as the standard assignment. Another important factor for the performance of an action recognition system is the feature extraction step. Recently, deep features obtained state-of-the-art results in many tasks, being also adopted for action recognition with competitive results over hand-crafted features. This work includes a pipeline to extract local deep features for videos using any available network as a black box and we show competitive results including the case when the network was trained for another task or another dataset. Our DA-VLAD encoding method outperforms the traditional VLAD and we obtain state-of-the-art results on UCF50 dataset and competitive results on UCF101 dataset.
Personal rating prediction for on-line video lectures using gaze information

Automatic prediction of personal preference for lecture video is becoming exceedingly important as the available volume of such content is expanding rapidly. Gaze information is believed to be an convenient and useful indicator of cognitive process due to its non-invasive characteristic. We build up a small-scale dataset of viewer's gaze during watching TED Talk videos, and propose a set of gaze features to predict personal preference. Besides the conventional gaze statistics features, we also propose new features that analyze the relationship between visual saliency and gaze points. The proposed features are proved to be effective in the evaluation of personal rating prediction and could achieve over 85% accuracy in the leave-one-out test and over 75% in the half-half train-test-split setting. To our best knowledge, this is the first work on rating prediction for video lectures using gaze.
Histograms of Motion Gradients for real-time video classification

Besides appearance information, the video contains temporal evolution, which represents an important and useful source of information about its content. Many video representation approaches are based on the motion information within the video. The common approach to extract the motion information is to compute the optical flow from the vertical and the horizontal temporal evolution of two consecutive frames. However, the computation of optical flow is very demanding in terms of computational cost, in many cases being the most significant processing step within the overall pipeline of the target video analysis application. In this work we propose a very efficient approach to capture the motion information within the video. Our method is based on a simple temporal and spatial derivation, which captures the changes between two consecutive frames. The proposed descriptor, Histograms of Motion Gradients (HMG), is validated on the UCF50 human action recognition dataset. Our HMG pipeline with several additional speed-ups is able to achieve real-time video processing and outperforms several well-known descriptors including descriptors based on the costly optical flow.
Interactive social, spatial and temporal querying for multimedia retrieval

We propose a scheme for faster and more effective retrieval of temporal, spatial and social multimedia from large collections. We define interactive multimedia queries that allow simultaneous query refinement on multiple search dimensions. User interaction techniques based on line and iconic sketches allow specifying queries based on the above definition. We prototype a multi-user travel media network and implement the proposed user interaction techniques for retrieving locomotion patterns of the users. The proposed queries facilitate easy input and refinement of queries, and efficient retrieval.
Mathematical error analysis of normal map compression based on unity condition

Normal maps play an important role in realistic 3D image rendering to express pseudo roughness of the surface. In normal map compression, z components are often eliminated and calculated from x and y components based on the unity condition to achieve high compression rate. However, there is no theoretical background of the validity of eliminating z components so far. In this paper, a mathematical mean square error (MSE) model of such normal map compression based on the unity condition is proposed. In addition, the boundary condition whether to eliminate or include the z components in compression for the better efficient encoding is demonstrated based on our error analysis model.
Video handover for retrieval in a ubiquitous environment using floor sensor data

A system for retrieving video captured in a ubiquitous environment is presented. Data from pressure-based floor sensors are obtained as a supplementary input together with video from multiple stationary cameras. Unsupervised data mining techniques are used to reduce noise present in floor sensor data. An algorithm based on agglomerative hierarchical clustering is used to segment footpaths of individual persons. Video handover is proposed and two methods are implemented to retrieve video and key frame sequences showing a person moving in the house. Users can query the system based on time and retrieve video or key frames using either of the handover techniques. We compare the results of retrieval using different techniques subjectively. We conclude with suggestions for improvements, and future directions.
Very high resolution image acquisition through image reconstruction from lower resolution images taken with multiple cameras

Some research institutes are elaborating plans to develop very high definition (VHD) visual media beyond the HDTV. Although a CCD camera with two million pixels has been developed for HDTV, the spatial resolution should be increased further to acquire VHD images. Reducing the pixel size is the most straightforward way to increase the spatial resolution, which makes a CCD imager much more sensitive to shot noise. To keep shot noise invisible on a monitor, there needs to be a limitation in the pixel size reduction. The current CCD technology has almost reached this limit. Therefore, a new approach is required to enhance the spatial resolution further beyond this limit. The authors present a new method for acquiring VHD images by processing and integrating multiple images taken simultaneously with different cameras.<<ETX>>
Marker-less human pose estimation and surface reconstruction using a segmented model

We propose a human motion tracking method for fast motion clips using synchronized multiple cameras. Our method is capable of extracting 3D articulated postures with 42 degrees of freedom through a sequence of visual hulls. We seek for the globally optimal solutions of the likelihood with the local memorization about the “fitness” of each body segment. Our method avoids the local minimum problem efficiently by mean combination and articulated combination of particles selected based on the weights of the different body segments. We deform the template surface model using the motion tracking data by linear blend skinning (LBS). Then we recover the details of the surface by fitting the deformed surface to 2D silhouettes.