nils-herrmann/clean_scirepeval huggingface dataset:
Detecting Dominant Motion Flows in Unstructured/Structured Crowd Scenes

Detecting dominant motion flows in crowd scenes is one of the major problems in video surveillance. This is particularly difficult in unstructured crowd scenes, where the participants move randomly in various directions. This paper presents a novel method which utilizes SIFT features’ flow vectors to calculate the dominant motion flows in both unstructured and structured crowd scenes. SIFT features can represent the characteristic parts of objects, allowing robust tracking under non-rigid motion. First, flow vectors of SIFT features are calculated at certain intervals to form a motion flow map of the video. 'ext, this map is divided into equally sized square regions and in each region dominant motion flows are estimated by clustering the flow vectors. Then, local dominant motion flows are combined to obtain the global dominant motion flows. Experimental results demonstrate the successful application of the proposed method to challenging real-world scenes.
Text detection in manga by combining connected-component-based and region-based classifications

As manga (Japanese comics) have become common content in many countries, it is necessary to search manga by text query or translate them automatically. For these applications, we must first extract texts from manga. In this paper, we develop a method to detect text regions in manga. Taking motivation from methods used in scene text detection, we propose an approach using classifiers for both connected components and regions. We have also developed a text region dataset of manga, which enables learning and detailed evaluations of methods used to detect text regions. Experiments using the dataset showed that our text detection method performs more effectively than existing methods.
Real-time image processing by using image compression sensor

We have been investigating an integration of sensing and compression on an image sensor. The compression sensor reduces the number of pixels in the image signal that has to be readout from the sensor. Therefore, the compression sensor can capture the images at the higher pixel rate which the traditional sensor can not handle. In this paper, we present a compression sensor which has 228/spl times/128 pixels. The processing circuits of the sensor can be operated at 5000 frames/second. We also describe the experimental results of two real-time image processing systems by using the compression sensor. They are a reconstruction circuit by using FPGA and a stereo image processing system for tracking of a moving object.
SVM is not always confident: Telling whether the output from multiclass SVM is true or false by analysing its confidence values

This paper presents an algorithm to distinguish whether the output label that is yielded from multiclass support vector machine (SVM) is true or false without knowing the answer. Such judgment is done only by the confidence analysis based on the pre-training/testing using the training data. Such true/false judgment is useful for refining the output labels. We experimentally demonstrate that the decision value difference between the top candidate and the second candidate is a good measure. In addition, a proper threshold can be determined by the pre-training/testing using only the training data. Experimental results using three standard image datasets demonstrate that our proposed algorithm can improve Matthews correlation coefficient (MCC) much better than simply thresholding the decision value for the top candidate.
Deformation-based data reduction of Time-Varying Meshes for displaying on mobile terminals

In this paper, a drastical compression method for Time-Varying Meshes (TVMs) is presented aiming at displaying on mobile terminals, where the resources are quite limited. In addition to our previous work of the skeleton-based temporal compression for TVMs, we have implemented spatial compression using 3D model morphing and mesh simplification. In addition, we have developed an interface for skeleton adjustment and modified the skeleton-based motion analysis so that rotation motion can be represented properly. As a result, we have achieved a very high compression rate of 1/5,000 and the rendering speed has been enhanced by six times as compred to our previous method. The feasibility of the algorithm has been confirmed on an actual cell phone.
An object-based non-blind watermarking that is robust to non-linear geometrical distortion attacks

This paper presents an object-based non-blind watermarking technique that is robust to non-linear geometrical distortion attacks. This has been one of the most challenging problems for copyright protection of digital content because it is difficult to estimate the distortion parameters for the embedded blocks. In our proposed scheme, the locations used to embed the watermark information are memorized and detected robustly by using relative coordinates with respect to the scale-invariant feature transform (SIFT) feature points. Experimental results with 64-bit watermark embedding demonstrated that the watermark detection performance was improved from 57% to 82% on average, even after nonlinear geometrical attacks such as waving, imploding, swirling, and seam carving.
Real-time objects tracking by using smart image sensor and FPGA

We have been investigating the integration of sensing and compression on an image sensor. The compression sensor reduces the number of pixels in the image signal that have to be readout from the sensor. We present a real-time object tracking system using a compression sensor, which has 128/spl times/128 pixels, and FPGA. By using this system, several moving objects can be extracted and tracked independently at 1200 frames/second. We also describe real-time depth estimation by using binocular compression sensors. We show some results obtained by the system.
Motion Segmentation for 3D Video Based on Spherical Registration

In this paper, a highly accurate motion segmentation technique for 3D video is presented. In conventional approaches, motion of the objects was analyzed using shape feature vectors extracted 3D video frames. This was because it was very difficult to locate and track feature points in the objects in the 3D space due to that fact that the number of vertices and connection vary each frame. In this study, we developed an algorithm to analyze the objects' motion in the 3D space using the spherical registration based on the iterated closest point algorithm. Rough motion tracking is conducted and the degree of motion is robustly calculated by this method. Although the approach is straightforward, much better motion segmentation results than the conventional approaches are obtained by yielding such high precision and recall rates as 95% and 92% on average.
Measurement and evaluation of comfort levels of apartments using IoT sensors

We developed IoT sensors for the evaluation of the comfort levels of real estate building properties. Our IoT sensor can measure temperature, humidity, illuminance, and acceleration and can record audio and video. We conducted proof-of-concept experiments by putting the sensors in six apartments in three buildings, and we succeeded in evaluating the difference in comfort levels of the rooms. For example, we revealed that an upper-floor apartment received sunshine for two more hours than a lower-floor apartment even though the two apartments were in the same building. Additionally, we found that in apartment buildings next to a railroad or an expressway, there is little noise if they were facing away from the source of the noise. Such information would help us to avoid having prejudiced evaluation to apartments and to have more fair opinions.
Content-Based Cross Search for Human Motion Data using Time-Varying Mesh and Motion Capture Data

This paper describes a content-based cross search scheme for two kinds of three-dimensional (3D) human motion data: time-varying mesh (TVM) and motion capture data. TVM is a sequence of 3D mesh models made for real-world 3D objects. TVM is generated using multiple-view images taken with multiple cameras. Since TVM can record shape, color, and motion of the real-world 3D objects, it has been drawing a lot of attention these days. In order to realize practical archiving systems for TVM, efficient retrieval systems are indispensable. The retrieval systems for TVM developed so far are based on query-by-example. This means additional TVM generation is required for constructing queries, which is computationally demanding and time consuming. On the other hand, motion capture systems are widely used to capture 3D human motion. However, the data structure is very different from that of TVM. Therefore, the two kinds of 3D human motion data are incompatible to each other. In this paper, we present a retrieval system that enables retrieving TVM using motion capture data as queries and vice versa using the modified shape distribution algorithm.
3D pose estimation in high dimensional search spaces with local memorization

In this paper, a stochastic approach for extracting the articulated 3D human postures by synchronized multiple cameras in the high-dimensional configuration spaces is presented. Annealed Particle Filtering (APF)  seeks for the globally optimal solution of the likelihood. We improve and extend the APF with local memorization to estimate the suited kinematic postures for a volume sequence directly instead of projecting a rough simplified body model to 2D images. Our method guides the particles to the global optimization on the basis of local constraints. A segmentation algorithm is performed on the volumetric models and the process is repeated. We assign the articulated models 42 degrees of freedom. The matching error is about 6% on average while tracking the posture between two neighboring frames.
Timing in systolic systems with variable minimum connection delays

Mapping an originally delay-free dataflow graph onto a systolic hardware consisting of various programmable ALUs (arithmetic logic unit) and a programmable network connecting them will introduce variable extra delays between the processing nodes. To ensure successful mapping it will usually be necessary to increase some of the delays. An algorithm is introduced for calculating these delays and the position to add them. This algorithm was implemented in PicPEn, a programming environment created for the Picot system.<<ETX>>
Use of steerable viewing window (SVW) to improve the visual sensation in face to face teleconferencing

We propose a method of using face direction information to make the local participant's video display of a video conferencing environment change according to his direction of view, in order to give him/her (the local participant) an improved visual sensation. We have named this method "steerable viewing window" (SVW). We also discuss an image processing method used to detect a person's face direction, without using special head mounted equipment. Further, we propose a method of reducing the data rate for transmission by coding the area around the gaze center with high resolution and the remaining with a lesser resolution. This method of saving bytes in video conferencing is named line of gaze centered coding (LOGCC).<<ETX>>
A multiple person eye contact (MPEC) teleconferencing system

A novel method of providing eye-contact for multiple participants in a teleconferencing system is presented. This method proposes a simple idea to fulfil the requirement of eye contact, for two or more local and remote participants simultaneously, by using half mirrors and cameras placed at the common points of the extended lines of gaze of each participant. In contrast to the commonly available one to one eye contact video phone systems, which also use half mirrors, the proposed system has the advantage of serving more than one person at a given location, hence named the multiple person eye contact (MPEC) teleconferencing system. We observed the resultant images by using a preliminary experiment and also building a prototype system. The results from both of these experiments proved the merits of the proposed idea.
A novel approach for combined rotational and translational motion estimation using Frame Projection Warping

This paper introduces a novel video stabilization technique for combined rotational and translational motion using integral frame projections. In the proposed Frame Projection Warping (FPW) method, the normalized intensity projection curves of two consecutive frames are warped using dynamic time warping, to get the relative shift between them. Rotational and vertical motion estimation involves partitioning of frame in to two halves and their corresponding estimated motions are then utilized for respective rotational angle and vertical shift estimation. This technique uses the human perception for analyzing the rotation in terms of vertical displacement of two halves of frame. The proposed technique is tested over various hand recorded videos. The results show better performance of FPW over various existing intensity based techniques. This technique also gives better accuracy in case of frame blurring, which is a serious cause of wrong motion estimation. The performance of the proposed technique is measured in terms of interframe transformation fidelity and processing time.
New method of on-sensor A/D conversion

We describe a CMOS image sensor with column parallel Analog to Digital Conversion circuits. In this method, we use small 1 bit comparator which detects each bit during integration 8 bit value is determined when the integration is finished. Consequently, both the A/D conversion and the integration of the pixel value can be done in parallel. We have designed a prototype by using column parallel architecture. The prototype outputs analog value and 8 bit digital value of pixel data. It has 32/spl times/16 pixels and digital memory for each pixel. We describe the processing scheme of our ADC and the circuit and layout design of the prototype. We show results of some experiments.
PQTable: Fast Exact Asymmetric Distance Neighbor Search for Product Quantization Using Hash Tables

We propose the product quantization table (PQTable), a product quantization-based hash table that is fast and requires neither parameter tuning nor training steps. The PQTable produces exactly the same results as a linear PQ search, and is 102 to 105 times faster when tested on the SIFT1B data. In addition, although state-of-the-art performance can be achieved by previous inverted-indexing-based approaches, such methods do require manually designed parameter setting and much training, whereas our method is free from them. Therefore, PQTable offers a practical and useful solution for real-world problems.
Robust watermark extraction using SVD-based dynamic stochastic resonance

In this paper, a novel dynamic stochastic resonance (DSR)-based non-blind watermark extraction technique has been proposed for robust extraction of a grayscale watermark. The watermark embedding has been carried out using singular value decomposition (SVD). Dynamic stochastic resonance has been strategically used to improve the robustness of the extraction algorithm by utilizing the noise added during attacks itself. Resilience of this technique to attacks has been tested in the presence of various noise, geometrical, enhancement, compression and filtering attacks. Using the DSR-based proposed extraction algorithm, a very robust extraction of watermark can be done without trading-off with visual quality of the watermarked image. Performance of the proposed technique has also been compared with the plain SVD-based and hybrid DCT-SVD based technique and is found to give better performance.
The log-normal distribution of the size of objects in daily meal images and its application to the efficient reduction of object proposals

In general, object-detection methods apply classifiers to pre-calculated object proposals. It is therefore important to minimize the number of proposals to achieve computational efficiency. In this paper, we show that the region size for food objects in recorded images of daily food follows a lognormal distribution, which is different from the distribution for widely used datasets collected by querying the names of dishes. We explain this characteristic using Gibrat's law, and construct a model for the region-size distribution of objects in images. We applied the model to the filtering of object proposals generated by selective search and edge boxes. We obtained a significant reduction of 40.6% in the number of hypotheses compared with a conventional selective search, despite a decrease of only 0.007 in the Mean Average Best Overlap.
Iterative reconstruction of an all-focused image by using multiple differently focused images

In this paper, we propose a novel method of all-focused image acquisition using multiple differently focused images. Based on the assumption that depth of the scene changes stepwise, we derive a formula for reconstruction between the desired all-focused image and multiple acquired images; we can reconstruct the all-focused image by iterative use of the formula. We also introduce coarse-to-fine estimation of point spread functions of the acquired images. We show we can reconstruct an all-focused image for a natural scene.
On sensor image compression for high pixel rate imaging: pixel parallel and column parallel architectures

We propose a novel concept of an integration of compression and sensing in order to enhance performance of an image sensor. By integrating the compression function on the sensor plane, the image signal that has to be read out from the sensor is significantly reduced. Thus, the integration can consequently increase the pixel rate of the sensor. The compression scheme we make use of is conditional replenishment that detects and encodes moving areas. In this paper, we discuss design and implementation of two architectures for on-sensor compression. One is a pixel parallel approach and the other is a column parallel approach. We describe both approaches and design and a prototype of pixel parallel architecture.
Depth Estimation Using an Infrared Dot Projector and an Infrared Color Stereo Camera

This paper proposes a method of estimating depth from two kinds of stereo images: color stereo images and infrared stereo images. An infrared dot pattern is projected on a scene by a projector so that infrared cameras can capture the scene textured by the dots and the depth can be estimated even where the surface is not textured. The cost volumes are calculated for the infrared and color stereo images for each frame and are extended in the time direction to define a spatiotemporal cost volume (st-cost volume). We also extend the cost volume filter in the time direction by modifying the cross-based local multipoint filter (CLMF) and applying it to the st-cost volumes in order to restrain flicker on the time-varying depth maps. To get a reliable cost volume, the infrared and color st-cost volumes are integrated into a single cost volume by selecting the cost of either the infrared or the color st-cost volumes according to the size of the adaptive kernel used for the CLMF. Then, a graphcut is executed on the cost volume in order to estimate the disparity robustly even when the baselines of the stereo cameras are set wide enough to ensure spatially high resolution in the depth direction and the shapes of blocks are deformed by the affine transformation. A 2D graphcut is executed on each scan line to reduce the processing time and memory consumption. We experimented with the proposed method using infrared color stereo data sets of scenes in the real world and evaluated its effectiveness by comparing it with other recent stereo matching methods and depth cameras.
How competitive are you: Analysis of people's attractiveness in an online dating system

An increasing number of people are using dating websites to search for their life partners. This leads to the curiosity of how attractive a specific person is to the opposite gender on an average level. We propose a novel algorithm to evaluate people's objective attractiveness based on their interactions with other users on the dating websites and implement machine learning algorithms to predict their objective attractiveness ratings from their profiles. We validate our method on a large dataset gained from a Japanese dating website and yield convincing results. Our prediction based on users' profiles, which includes image and text contents, is over 80% correlated with the real values of the calculated objective attractiveness for the female and over 50% correlated with the real values of the calculated objective attractiveness for the male.
Computational image sensor for on sensor compression

We propose a novel integration of image compression and sensing in order to enhance the performance of an image sensor. By integrating a compression function onto the sensor focal plane, the image signal to be read out from the sensor is significantly reduced and the pixel rate of the sensor ran consequently be increased. The potential applications of the proposed sensor are in high pixel-rate imaging, such as high frame-rate image sensing and high-resolution image sensing. The compression scheme we employ is a conditional replenishment, which detects and encodes moving areas. In this paper, we introduce two architectures for on-sensor compression; one is the pixel parallel approach and the other is the column parallel approach. We prototyped a VLSI chip of the proposed sensor based on the pixel parallel architecture. We show the design and describe the results of the experiments obtained by the prototype chip.
Construction of large-scale virtual environment by fusing range data,texture images, and airborne altimetry data

Construction of large-scale virtual environment is gaining more attentions for its applications in virtual mall, virtual sightseeing, tele-presence, etc. We introduce a framework to construct realistic large-scale virtual environment by fusing range data, texture images, and airborne altimetry data. First, realistic and high-precision 3-D models of buildings are created from range data and texture images. Next, rough 3-D models of buildings in wide area are created using altimetry data acquired by airborne laser profiler. Finally, these models are integrated to build a largescale realistic walk-through system. This paper describes the proposed system, part of its implementations, as well as future works that still have to be done.
Key Frame Extraction in 3D Video by Rate-Distortion Optimization

3D video, which consists of a sequence of 3D mesh models, can provide detailed 3D information both in spatial and temporal domain. In this paper, a key frame extraction method has been developed to summarize 3D video by rate-distortion optimization. For this purpose, we introduce an effective feature vector extraction algorithm from 3D video. Prior to key frame extraction, shot detection is performed using the feature vectors as a pre-processing. Then, a rate-distortion (R-D) curve is generated in each shot, where the locations of key frames are optimized. Lastly, R-D trade-off can be achieved by optimizing a cost function with a Lagrange multiplier. Our experimental results show the extracted key frames are compact and faithful to original 3D video