nils-herrmann/clean_scirepeval huggingface dataset:
Time-Varying Mesh Compression Using an Extended Block Matching Algorithm

Time-varying mesh, which is attracting a lot of attention as a new multimedia representation method, is a sequence of 3-D models that are composed of vertices, edges, and some attribute components such as color. Among these components, vertices require large storage space. In conventional 2-D video compression algorithms, motion compensation (MC) using a block matching algorithm is frequently employed to reduce temporal redundancy between consecutive frames. However, there has been no such technology for 3-D time-varying mesh so far. Therefore, in this paper, we have developed an extended block matching algorithm (EBMA) to reduce the temporal redundancy of the geometry information in the time-varying mesh by extending the idea of the 2-D block matching algorithm to 3-D space. In our EBMA, a cubic block is used as a matching unit. MC in the 3-D space is achieved efficiently by matching the mean normal vectors calculated from partial surfaces in cubic blocks, which our experiments showed to be a suboptimal matching criterion. After MC, residuals are transformed by the discrete cosine transform, uniformly quantized, and then encoded. The extracted motion vectors are also entropy coded after differential pulse code modulation. As a result of our experiments, 10%-18% compression has been achieved.
View-Based Web Page Retrieval using Interactive Sketch Query

We propose a novel view-based Web page retrieval system that enables a user to search Web pages using a visual query, namely the user's freehand sketch. We believe the proposed method will suit retrieval from a set of web pages such as a user's local browsing history. The system aims to help the user revisit a particular Web page without using query words. Using color signature features and Earth-Mover's distance, the system evaluates the similarity between web pages and the user's sketch drawn via the GUI of the system. In order to accelerate the interaction, the results of the similarity evaluations are shown immediately after the user draws each stroke of the sketch, with the results being interactively reordered. Experiments using our prototype system showed that users find their target pages after only a few strokes. Experimental results for inexperienced users showed that 71% of search tasks were completed within one minute, using the prototype system. The median time for the tasks was 40 seconds.
Fast Face Model Reconstruction and Synthesis Using an RGB-D Camera and Its Subjective Evaluation

It is difficult to show a frontal face in video chatting because there is a gap between a display and a camera. We propose a method for real-time face reorientation by creating a 2.5-D face model from a single RGB-D camera and synthesizing the rotated face model with the original face image. Our method uses two kinds face models complementarily: a point cloud based model and a generic face model fitted to the user. We conducted subjective evaluation and confirmed the validity of our proposed system.
Wearable Video Retrieval and Navigation System using GPS Data

We have investigated a wearable video system to capture our experiences by a wearable camera, microphone and some sensors including GPS receiver, etc. GPS receiver was used to record the user’s tracks and identify the location of user’s experiences by latitude and longitude coordinates. GPS receiver can also extract time, speed and direction that are benefit contexts for wearable video navigation. Thus, we attempt to use available data from GPS to extract the key events from wearable video. In this paper, we propose a wearable video retrieval and navigation by using GPS data. Our system provides two types of navigation functions including key frame based navigation and location based navigation. Key frame based navigation is used to extract the key frames from wearable video to summarize the video contents. Location based navigation can be used to retrieve the wearable video based on user’s tracks by using a map. Both of them have the relationship and interfaces to present each other. We demonstrate the key frame extraction techniques based on moving speed detection, directional change detection and the comparison between time and distance sampling. The key frame extraction techniques were evaluated based on the performance to detect landmarks and desired events. The results showed that the navigation system using GPS data is helpful to retrieve and extract the key events during traveling scenes in daily life.
High level activity annotation of daily experiences by a combination of a wearable device and Wi-Fi based positioning system

Many people would like to record and manipulate their experiences effectively. However, efficient summarization to show ldquowhat, when and whererdquo we did in our daily lives is still an open issue in life log applications. In conventional approaches, many sensors were attached to a human body to solve this problem. However, this is not practical for daily use. In this paper, we propose a simple solution in which a user wears two devices: a single life-logging device SenseCam  hung by neck and a Wi-Fi enabled PDA. The location data and low-level activity are analyzed by a Wi-Fi based positioning system. Then, high-level activity classification is conducted using the wearable device along with a refinement process considering the time consistency. Experimental results demonstrated high recall and precision rates as much as 81% and 85% respectively, on average.
Review-Based Service Profiling and Recommendation

Finding desired services (e.g., restaurants) from a number of candidates is a difficult task. We usually look at the scores given by other customers or read their reviews for decision making, but it is time consuming. This paper presents review-based service recommendation using both a bag-of-words model and a skip-gram-based model. Namely, instead of checking the scores or review comments, a review is given to the system as a query, and the services to which similar reviews are given are recommended. We evaluated our algorithms by using Yelp's Academic Dataset. The experimenatl results show that the percentage of finding the corresponding restaurants as the top-1 candidate was 25% and that in the top-10 ranking was 51%.
Summarization of wearable videos using support vector machine

Auto-summarization of video contents has become an important topic following the growing amount of multimedia contents. Researches in this area have shown the effectiveness of low-level video and audio features in categorizing video contents. The use of brainwaves to reflect personal interests is also proven to be practical. In this paper, we model the relationship between audio/video features and brainwaves (/spl alpha/-waves) using the support vector machine (SVM). Based on the SVM model, we summarized wearable videos by personal interests, using only low-level video and audio features. Here we define "wearable videos" as continuous recordings of personal experiences using wearable video camera and computer. Our experiment results showed over 90% of accuracy on summarization of a 25-minute video clip with an SVM model created by another resembling 25-minute video clip.
Wearable imaging system for summarizing personal experiences

Digitization of lengthy personal experiences would be made possible by constant recording using wearable video cameras. It is conceivable that the resulting amount of video content would be extraordinarily large. In order to retrieve and browse the desired scenes, a vast amount of video would need to be organized with structural information. In this paper, we attempt to develop a "wearable imaging system" that is capable of constantly capturing data, not only from a wearable video camera, but also from various kinds of sensors, such as a GPS, an accelerometer and a gyro sensor. The data from these sensors are appropriately extracted and processed by hidden Markov model (HMM) to achieve efficient video retrieval and browsing.
Mutual Information in 3D Video

3D video, which is captured by multiple synchronized cameras and consists of a sequence of mesh models, is becoming a hot topic recently. Similarity measure between frames is a powerful tool in video analysis. In this paper, an effective similarity measure based on mutual information is proposed for 3D video. Only vertex positions in mesh models are utilized in our mutual information. The mutual information is calculated based on the distance field in volumetric models, which are converted from mesh models. Our experiments demonstrate the mutual information can reflect the motion of the object in 3D video very well as a similarity measure.
Real-time arbitrary-view acquisition system by using random access IBR camera array

Image-based rendering (IBR) is a powerful tool for synthesizing arbitrary-views of a real scene. So far, almost all the IBR researches are limited to static scenes because of the difficulty in simultaneously capturing a dynamic scene by a number of cameras. Although IBR needs many images of the real scene from different view positions, only a small number of pixels per each image are used for the synthesis. We propose a capturing system for real-time IBR for dynamic scenes, which uses special random access image sensors. In the proposed system, image sensors selectively outputs pixels that are required for synthesis and the selection of the pixels are dynamically changed according to the position of the virtual view. We designed and prototyped CMOS random access image sensors for the real-time IBR. By using sixteen of those sensors, we build a random access IBR camera array system. In this paper, we describe the design and the experimental results of the camera array system.
An image sensor for on-sensor-compression

We present a novel image sensor on which image signal can be compressed. Since image signal is compressed on an imager plane by making use of parallel nature of image signals, the amount of signal read out from the imager can be significantly reduced. Thus, the proposed sensor can be potentially applied to high pixel rate cameras and processing systems which require very high speed imaging and very high resolution real time imaging; the very high bandwidth is the fundamental limitation for feasibility of those high pixel rate sensors and processing systems. Analog circuits have been designed both for processing in each pixel and for controlling entire data rate. A first prototype of a VLSI chip has been fabricated. Some results of the examination are shown in this paper.
Affective Audio-Visual Words and Latent Topic Driving Model for Realizing Movie Affective Scene Classification

This paper presents a novel method for movie affective scene classification that outputs the emotion (in the form of labels) that the scene is likely to arouse in viewers. Since the affective preferences of users play an important role in movie selection, affective scene classification has the potential to develop more attractive user-centric movie search and browsing applications. Two main issues in designing movie affective scene classification are considered. One is “how to extract features that are strongly related to the viewer's emotions”, and the other is “how to map the extracted features to the emotion categories”. For the former, we propose a method to extract emotion-category-specific audio-visual features named affective audio-visual words (AAVWs). For the latter issue, we propose a classification model named latent topic driving model (LTDM). Assuming that viewers' emotions are dynamically changed by the movie scene sequences, LTDM models emotions as Markovian dynamic systems driven by the sequential stimuli of the movie content. Experiments on 206 movie scenes extracted from 24 movie titles and the corresponding labels of eight emotion categories given by 16 subjects show that our method outperforms conventional approaches in terms of the subject agreement rate.
Food Log by snapping and processing images

We present the current status of FoodLog, a multimedia Internet application that enables easy capture and archival of information regarding our daily meals. The primary purpose of FoodLog is to facilitate dietary management support with minimum manual recording of information. It analyzes image archives that belong to a user to identify images of meals. Further image analysis determines the nutritional composition of these meals and stores the data to form a log. The user can view the data from this log in different formats, and also edit the data to correct any mistakes that occurred during image analysis. This application was recently opened to the public, and had accumulated approximately 25000 images during the first two months since its launch. We present the current status of this application, and discuss our future plans to extend it to allow interaction between users and more effective dietary management.
You Will Succeed or Not? Matching Prediction in a Marriage Consulting Service

On online dating websites, a match is accomplished only when two users have mutual interests. Directed contact is a sign of users' interest and our research intends to provide recommendations by predicting whether two users will have reciprocal contact. We learn matching patterns by supervised machine learning and evaluate our framework on a large dataset extracted from a popular dating website. We propose a novel computational pipeline to extract features from both users' profiles and dating social networks to know users' basic information and their preferences. Our results show that both profile-based features and graph-based features can be used to effectively predict whether two users are likely to make reciprocal contact. Best results are achieved when using all of the features as the input to a Support Vector Machine (SVM) and its Matthews correlation coefficient is 0.372. We also prove that users' facial features provide effective information about them and can be used to predict whether two users match each other.
Creation of an Electronic Chronicle for a Ubiquitous Home: Sensing, Analysis and Evaluation

A system for continuous capture and retrieval of multimedia data from a home-like ubiquitous environment is presented. A large number of cameras and microphones are used for video and audio acquisition. Pressure based sensors mounted on the house floor are used to capture footsteps as context data. Both context and content are analyzed to create indices for efficient retrieval and summarization of the video and audio data. The system can be queried based on date and time, location, persons, sounds and basic activity in terms of footsteps. Key frame extraction adaptive to the activity of a person can be used to summarize the video. The system was evaluated by means of a real-life experiment where a family lived in the house and later used the system to retrieve experiences that they had during their stay. The results of this experiment are reported and discussed. The paper concludes with suggestions for possible future directions.
Efficient Optimization of Convolutional Neural Networks Using Particle Swarm Optimization

This work presents methods to automatically find optimal parameter settings for convolutional neural networks (CNNs) by using an evolutionary algorithm called particle swarm optimization (PSO). Even though the parameter space is extremely large (> 10 20), we experimentally show that a better parameter setting can be found for Alexnet configuration for five different image datasets. We have also developed two candidate pruning algorithms for efficient evolutionary process. In the experiments, we achieved 0.7-5.7% improvements from the original parameter sets in Caffe, while requiring only 2-4% of processing cost of the naive PSO-based approach.
Analysis and synthesis of facial image sequences in model-based image coding

This paper proposes new methods for analyzing image sequences and updating textures of the three-dimensional (3-D) facial model. It also describes a method for synthesizing various facial expressions. These three methods are the key technologies for the model-based image coding system. The input image analysis technique directly and robustly estimates the 3-D head motions and the facial expressions without any two-dimensional (2-D) entity correspondences. This technique resolves the 2-D correspondence mismatch errors and provides quality reproduction of the original images by fully incorporating the synthesis rules. To verify the analysis algorithm, the paper performs quantitative and subjective evaluations. It presents two methods for updating the texture of the facial model to improve the quality of the synthesized images. The first method focuses on the facial parts with large change of brightness according to the various facial expressions for reducing the transmission bit rates. The second method focuses on all changes of brightness caused by the 3-D head motions as well as the facial expressions. The transmission bit rates are estimated according to the update methods. For synthesizing the output images, it describes rules that simulate the facial muscular actions because the muscles cause the facial expressions. These rules more easily synthesize the high-quality facial images that represent the various facial expressions.
Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos

We introduce Spatio-Temporal Vector of Locally Max Pooled Features (ST-VLMPF), a super vector-based encoding method specifically designed for local deep features encoding. The proposed method addresses an important problem of video understanding: how to build a video representation that incorporates the CNN features over the entire video. Feature assignment is carried out at two levels, by using the similarity and spatio-temporal information. For each assignment we build a specific encoding, focused on the nature of deep features, with the goal to capture the highest feature responses from the highest neuron activation of the network. Our ST-VLMPF clearly provides a more reliable video representation than some of the most widely used and powerful encoding approaches (Improved Fisher Vectors and Vector of Locally Aggregated Descriptors), while maintaining a low computational complexity. We conduct experiments on three action recognition datasets: HMDB51, UCF50 and UCF101. Our pipeline obtains state-of-the-art results.
A computational image sensor with pixel-based integration time control

We have been investigating a computational sensor which controls the integration time of every pixel independently. Because the integration time is controlled, higher temporal resolution and wider dynamic range can be achieved. We present a new adaptive integration time image sensor which has 128/spl times/64 pixels. We adopt a column parallel architecture to design the prototype chip. The scheme to control integration time is extended, and pixel pitch, processing speed and power consumption are much improved in comparison with our previous prototype. We show some experimental results obtained with the prototype.
Interactive segmentation for manga using lossless thinning and coarse labeling

Manga (Japanese comics) are popular all over the world. Applications using manga, such as automatic layout suggestion and character composition, have been proposed recently. To implement these applications, important but time-consuming manga segmentation is required. We propose a scribble-based interactive segmentation method for manga images. Based on Smart Scribbles, we propose (1) lossless thinning, which enables the handling of complex raster images such as manga, and (2) coarse labeling, which drastically reduces the computational cost of labeling and enables accurate segmentation in real time. Our experimental results show that the proposed method efficiently produces much more accurate results than does the Quick Selection Tool in Adobe Photoshop1® or the connected-component labeling method.
Repositioning the salient region of videos by using active illumination

The use of networked light bulbs is proposed in order to modify the focus of attention, defined based on salience maps, of a video being displayed on a screen. The proposed system was evaluated by using four networked lamps positioned behind a television display. The lamps were controlled by the system according to the visual features of the video. By using a camera, the salience of the room illuminated by the smart bulbs was recorded and compared against the salience of the room when the lamps were not used. The results showed that changes in color and intensity of the environmental illumination were able to modify the shape of the salience maps of the displayed videos. Two modes of operation are proposed: salience cancellation and salience enhancement.
Fast and Robust Motion Tracking for Time-Varying Mesh Featuring Reeb-Graph-Based Skeleton Fitting and its Application to Motion Retrieval

In this paper, an algorithm for motion extraction from time-varying mesh (TVM) is proposed. TVM is a sequence of 3D models made for real-world dynamic objects. In TVM, detailed information of 3D objects such as shape, color, and motion is stored. Therefore, TVM is an attractive technology for the next-generation multimedia. However, the management of TVM database for archiving and retrieving is a significant problem because it is difficult to locate and track feature points in TVM. For efficient and effective archive and retrieval systems, motion extraction and tracking is essential. In our approach, skeletons are extracted from TVM using Reeb graph, and motion tracking is achieved by fitting a reference skeleton to the others. We defined a geodesic function using principal component analysis (PCA) for fast and noiseless skeleton extraction. Robust fitting has been realized by an end node tracking strategy. In addition, we have developed an efficient motion retrieval system compatible to conventional motion capture (mocap) data using the extracted skeleton. As a result, low-cost query generation using mocap systems and cross search between TVM and mocap data have been achieved.
Image processing based approach to food balance analysis for personal food logging

Food images have been receiving increased attention in recent dietary control methods. We present the current status of our web-based system that can be used as a dietary management support system by ordinary Internet users. The system analyzes image archives of the user to identify images of meals. Further image analysis determines the nutritional composition of these meals and stores the data to form a Foodlog. The user can view the data in different formats, and also edit the data to correct any mistakes that occurred during image analysis. This paper presents detailed analysis of the performance of the current system and proposes an improvement of analysis by pre-classification and personalization. As a result, the accuracy of food balance estimation is significantly improved.
Inverse filters for reconstruction of arbitrarily focused images from two differently focused images

This paper describes a novel filtering method to reconstruct an arbitrarily focused image from two differently focused images. Based on the assumption that image scene has two layers-foreground and background-, two differently focused images are used as inputs, one of which is focused on the foreground and the other is focused on the background. The linear equation that holds between these images and the desired image, which is derived from their imaging models, can be formulated as an image restoration problem. This paper shows that the solution of this problem exists as an inverse filter and the desired image can be reconstructed only by the linear filters. As a result, fast reconstruction with high accuracy can be achieved. Experiments using real images are shown.