nils-herrmann/clean_scirepeval huggingface dataset:
POP: Person Re-identification Post-rank Optimisation

Owing to visual ambiguities and disparities, person re-identification methods inevitably produce sub optimal rank-list, which still requires exhaustive human eyeballing to identify the correct target from hundreds of different likely-candidates. Existing re-identification studies focus on improving the ranking performance, but rarely look into the critical problem of optimising the time-consuming and error-prone post-rank visual search at the user end. In this study, we present a novel one-shot Post-rank Optimization (POP) method, which allows a user to quickly refine their search by either "one-shot" or a couple of sparse negative selections during a re-identification process. We conduct systematic behavioural studies to understand user's searching behaviour and show that the proposed method allows correct re-identification to converge 2.6 times faster than the conventional exhaustive search. Importantly, through extensive evaluations we demonstrate that the method is capable of achieving significant improvement over the state-of-the-art distance metric learning based ranking models, even with just "one shot" feedback optimisation, by as much as over 30% performance improvement for rank 1 re-identification on the VIPeR and i-LIDS datasets.
Two-stream binocular network: Accurate near field finger detection based on binocular images

Fingertip detection plays an important role in human computer interaction. Previous works transform binocular images into depth images. Then depth-based hand pose estimation methods are used to predict 3D positions of fingertips. Different from previous works, we propose a new framework, named Two-Stream Binocular Network (TSBnet) to detect fingertips from binocular images directly. TSBnet first shares convolutional layers for low level features of right and left images. Then it extracts high level features in two-stream convolutional networks separately. Further, we add a new layer: binocular distance measurement layer to improve performance of our model. To verify our scheme, we build a binocular hand image dataset, containing about 117k pairs of images in training set and 10k pairs of images in test set. Our methods achieve an average error of 10.9mm on our test set, outperforming previous work by 5.9mm (relatively 35.1%).
A hand gesture based interactive presentation system utilizing heterogeneous cameras

In this paper, a real-time system that utilizes hand gestures to interactively control the presentation is proposed. The system employs a thermal camera for robust human body segmentation to handle the complex background and varying illumination posed by the projector. A fast and robust hand localization algorithm is proposed, with which the head, torso, and arm are sequentially localized. Hand trajectories are segmented and recognized as gestures for interactions. A dual-step calibration algorithm is utilized to map the interaction regions between the thermal camera and the projected contents by integrating a Web camera. Experiments show that the system has a high recognition rate for hand gestures, and corresponding interactions can be performed correctly.
FEC-based scalable multiple description coding for overlay network streaming

We consider the problem of distributing video data from a server to a population of interested clients. Deploying overlay network with application-layer multicast, we propose a FEC-based scalable multiple description coding (SMDC) scheme to arrive at the solution to the problem. Without the performance penalty on each receiver, our SMDC can achieve optimal performance of each client with different network conditions as simply as additional assembling and truncating operations. In the mean time, a R-D allocation algorithm is presented to decide the rates of the source and channel. Simulation shows that our SMDC have better quality for heterogeneous receivers than traditional scheme.
A miniaturized wearable wireless hand gesture recognition system employing deep-forest classifier

This paper presents a wearable hand gesture recognition (HGR) system, which can decode the information from surface electromyography (sEMG) and micro-inertial measurement unit μ-IMU. With the cooperation between sEMG and IMU, the number of sEMG electrodes is reduced to 2 pairs without scarifying the accuracy and recognition range, which significantly shorten the distance to practical applications. For low-power and high-security concerns, a capacitive coupled body channel communication (CC-BCC) module is also implemented in the system for wireless communication. Last, a modified deep forest algorithm is employed to predict the gestures from the signal sources with high accuracy and robustness. Finally, 16 hand gestures include 10 dynamic and 6 static gestures are recognized on two different subjects, the proposed system can achieve 96% accuracy, and the prediction time for each sample is less than 6 ms.
Channel-adaptive error protection for scalable video over channels with bit errors and packet erasures

Video communication over channels with both bit errors and packet erasures is becoming increasingly important due to the emerging of wireless Internet. To have QoS provision at application level, product code is suited to correct the transmission errors occurring in such channels. In this paper, we presented a novel layered product code for scalable video streaming over the channels with both bit errors and packet losses. An end-to-end architecture is proposed to simultaneously address the error control, packetization and rate-distortion based bit allocation. Once the available network conditions are estimated, the unequal product codes are applied to different layers of scalable video. Specifically, each layer is first divided into blocks with unequal row channel code and unequal column channel code added. Then rate-distortion based bit allocation is proposed to determine the channel rates and the source rate so as to minimize the expected end-to-end distortion. The simulations demonstrated effectiveness of our proposed error protection scheme.
Anomaly detection in surveillance video using motion direction statistics

A novel approach for detecting anomaly in visual surveillance system is proposed in this paper. It is composed of three parts:(a) a dense motion field and motion statistics method, (b) one-class SVM for one-class classification, (c) motion directional PCA for feature dimensionality reduction. Experiments demonstrate the effectiveness of proposed algorithm in detecting abnormal events in surveillance video, while keeping a low false alarm rate. Moreover, it works well in complicated situation where the common tracking or detection module won't work.
Error protection for scalable image over 3G-IP network

Digital media, like image and video, over third-generation wireless networks is a challenging task because the wireless networks present not only packet loss, but also bit errors. To address this problem, this paper proposes a novel error protection scheme for scalable image over 3G-IP networks. Taking into consideration the scalable nature of the image data, error protection is provided by layered product channel codes to mitigate the effect of the packet loss and bit errors. Meanwhile, rate-distortion optimization is performed to determine the protection levels of both the row channel codes and the column codes so as to minimize the expected end-to-end distortion. Simulation results demonstrate the effectiveness of our proposed approach.
Resource allocation with adaptive QoS for multimedia transmission over W-CDMA channels

This paper addresses the important issues of resource allocation and rate adaptation for multiple media, such as audio, video, email, and Web traffic, transmitted over W-CDMA (wideband code division multiple access) channel with adaptive QoS (quality of service) support. In order to have QoS support for different types of media, we develop an architecture combining the link layer with application layer controls. It consists of the following contributions: (1) an appropriate model to estimate the varying fading channel is proposed; (2) a hybrid delay-constrained ARQ (automatic repeat request) and UEP (unequal error protection) mechanism that dynamically adapt to the time-varying channel is presented to meet the QoS requirements for different applications; (3) a new resource allocation scheme that considers varying media characteristics is described to be adapted to changing bit error rate (BER) conditions. Simulation results demonstrate the effectiveness of our proposed scheme.
Topology based affine invariant descriptor for MSERs

This paper introduces a topology based affine invariant descriptor for maximally stable extremal regions (MSERs). The popular SIFT descriptor computes the texture information on a grey-scale patch. Instead our descriptor use only the topology and geometric information among MSERs so that features can be rapidly matched regardless of the texture in the image patch. Based on the ellipses fitting for the detected MSERs, geometric affine invariants between ellipses pair are extracted as the descriptors. Finally topology based voting selector is designed to achieve the best correspondences. Experiment shows that our descriptor is not only computational faster than SIFT descriptor, but also has better performance on wide angle of view and nonlinear illumination change. In addition, our descriptor shows a good result on multi sensor images registration.
Enhanced Sports Video Shot Boundary Detection Based on Middle Level Features and a Unified Model

The performance of shot boundary detection algorithms using low level features can hardly fulfill the requirement of automatic sports video analysis, which is a promising module in personal video recorders. Two kinds of middle level features are proposed in this paper to effectively enhance the shot boundary detection. One kind of the features is extracted from the projection of the dominant color mask; the other kind is extracted from the reliable block-based motion vectors. These novel features, together with the region color histograms feature, are integrated into a unified model which employs support vector machines to detect both cuts and gradual transitions. Experiments on diversified soccer video sequences demonstrate that our scheme outperforms the existing algorithms and is quite competent for the targeted applications.
The on-line viewing system of the distributed gigapixel image

The observation of the gigapixel image is the current research hotspot and how to view the image in short time is a big challenge. So in this paper we present an on-line viewing system of the distributed gigapixel image. We construct the MPI programming framework and use the MapReduce data processing model to stitch the subgraphs distributed in the different servers. Based on the size and location of the region of interest (ROI), only the relevant subgraphs will be transmitted and processed. Utilizing our on-line viewing system, users can observe the ROI of the gigapixel image in short time.
Iterative transductive learning for alpha matting

In this paper, we propose a matting algorithm based on iterative transductive learning (for short: ITM). To avoid over-smooth results of recent methods, we introduce the influence of unlabeled regions as well as the consistency of neighboring pixels to re-design the optimization for alpha matting. A novel asymmetric Laplacian matrix is also proposed to further relieve the over-smoothness. To optimize the matting problem, we adjust the constrain coefficients between the initialized alpha matte and the asymmetric Laplacian matrix iteratively to achieve accurate alpha mattes. Consequently, during the iteration, high confidence pixels maintain their refined alpha values, whereas low confidence ones are updated by their neighbors gradually. Experimental results demonstrate that our algorithm is more precise than many state-of-the-art methods in terms of the accuracy.
JPEG2000 based real-time scalable video communication system over the Internet

This paper presents a JPEG2000 based real-time scalable video communication system developed in Sony, named as "BEAM" video system. Two practical objectives are emphasized in this system: scalable video communication and real-time communication. We extend IETF RTP and RTSP streaming protocol to take care of scalable delivery of video data from a single layered coding data to heterogeneous devices and different resolution display such as HDTV, standard TV, PDA and mobile phone. Utilizing an advantage of low coding delay JPEG2000 codec, real-time ARQ (RT-ARQ) is proposed to resolve the QoS issues for real-time applications. In addition to this protocol, various network adaptive control techniques, including rate control and error control, are presented to achieve high quality video transmission.
Data level object detector adaptation with online multiple instance samples

In object detection, the offline trained detector's performance may be degraded in a particular deployed environment, because of the large variation of different environments. In this work, we propose a data level object detector adaptation method to new environments. By recording a small amount of offline data, it's fully compatible with offline training method and easy to implement. We re-derive an efficient MILBoost by eliminating line search in optimization and introduce it to collect online multiple instance samples, which don't require strict sample alignment. Experiment results with the human detector on public datasets illustrate the effectiveness of the proposed adaptation method. The adapted detector has good adaptation ability, while maintaining its generalization ability as well.
Live demonstration: A hand gesture recognition wristband employing low power body channel communication

This demonstration includes a hand gesture recognition (HGR) wristband, which can decoding the information in surface electromyography (sEMG) and micro-inertial measurement unit μ-IMU. With the complementary information from IMU, the wristband only needs two pairs of sEMG electrodes, which significantly improves the user's experiences. A capacitive coupled body channel communication (CC-BCC) module is also implemented in the system for wireless communication without antennas for low-power and security concerns. In this demonstration, visitors can intuitively understand the benefit and application of body channel communication for body area network.
Practical rate control for video over WLAN

Transmission Rate control is an important issue for video streaming applications, particularly for mobile consumer devices in wireless network. We propose two rate control algorithms to enhance the performance of 802.11 streaming, where practical problems are taken into account, including the RTT fluctuation and real packet loss events. Extended from our wired TFRC , the first scheme, named as congestion differentiated rate control (CDRC), predicts the network status based on RTT and packet loss events feedback from the receiver. The second scheme is based on the cross layer strategy, named as WiQ-X, where the frame error rate in PHY layer helps to estimate the channel status and adapt the sending rate. Practical experiments show that both of our schemes can achieve stable transmission and high quality of video streaming, especially in the bad channel conditions. KeywordsWireless LAN, rate control, congestion control, congestion differentiated, cross-layer structure
Mining Latent Attributes From Click-Through Logs for Image Recognition

Attribute-based image representation, which represents an image by projecting it into a space spanned by attributes, has attracted increasing attention from both computer vision and multimedia communities for its compactness and potential to bridge the semantic gap. While many works focus on learning attribute models and utilizing them in image recognition and retrieval, few touch on the problem of how to effectively construct a vocabulary of attributes, which is an essential part of effective attribute-based representation. Most existing approaches define the attribute vocabulary by human experts or through existing ontology, which is often limited in coverage of general concept space. In this paper, we propose automatically constructing the attribute vocabulary by mining latent topics from the click-through log of a commercial image search engine. These attributes are referred to as latent topic attributes (LTA), which take advantage of tens of millions of interactions between user submitted queries and images, thereby providing better coverage for the concept space than existing approaches. The mining of latent topics from the click log is formulated as a matrix factorization problem, and further improved by weighted terms-based matrix factorization to address the extreme sparsity of the click-through matrix. Both qualitative results of the mined LTA and quantitative results on the standard image recognition benchmark demonstrate the mined LTA's effectiveness.
PartBook for image parsing

Effective image parsing needs a representation that is both selective (to inter-class variations) and invariant (to intra-class variations). CodeBook from bag-of-visual-words representation addresses the invariance, and part-based models can potentially address the selectivity. However, existing part-based approaches either require expensive manual object-level labeling or make strong assumptions not applicable to real-world images. In this paper, we propose a PartBook approach that simultaneously overcomes the above two difficulties. Furthermore, we present an effective framework that integrates CodeBook and PartBook, which achieves both intra-class invariance and inter-class selectivity. Specifically, a set of candidate regions are first selected from heat map-like representations obtained by a SVM classifier trained for each category. Then the regions are clustered based on the dense matching-based similarity, and a part detector is learned from each cluster and further refined by utilizing a latent SVM. The learned PartBook summarizes the most representative mid-level patterns of each category, and can be readily used for image parsing tasks to identify not only objects but also different parts of an object. Extensive experimental results on real-world images show that the automatically learned parts are semantically meaningful, and demonstrate the effectiveness of ParkBook in image parsing tasks at different levels.
The shortest warping path based multiple images alignment

In this paper, we propose a method to align multiple images of the same category. Images with large variations are aligned via a smooth transition formed by some intermediate images. These intermediate images are found by shortest warping path algorithm on a directed complete graph. Moreover, the common regions in the images are discovered to further improve alignment performance. The experimental results show that our method is effective to map and align images of the same category but with large variations of appearance, shape and view.
You Lead, We Exceed: Labor-Free Video Concept Learning by Jointly Exploiting Web Videos and Images

Video concept learning often requires a large set oftraining samples. In practice, however, acquiring noise-free training labels with sufficient positive examples is very expensive. A plausible solution for training data collection is by sampling from the vast quantities of images and videos on the Web. Such a solution is motivated by the assumption that the retrieved images or videos are highly correlated with the query. Still, a number ofchallenges remain. First, Web videos are often untrimmed. Thus, only parts of the videos are relevant to the query. Second, the retrieved Web images are always highly relevant to the issued query. However, thoughtlessly utilizing the images in the video domain may even hurt the performance due to the well-known semantic drift and domain gap problems. As a result, a valid question is how Web images and videos interact for video concept learning. In this paper, we propose a Lead-Exceed Neural Network (LENN), which reinforces the training on Web images and videos in a curriculum manner. Specifically, the training proceeds by inputting frames of Web videos to obtain a network. The Web images are then filtered by the learnt network and the selected images are additionally fed into the network to enhance the architecture and further trim the videos. In addition, Long Short-Term Memory (LSTM) can be applied on the trimmed videos to explore temporal information. Encouraging results are reported on UCFIOl, TRECVID 2013 and 2014 MEDTest in the context ofboth action recognition and event detection. Without using human annotated exemplars, our proposed LENN can achieve 74.4% accuracy on UCFIOI dataset.
Learning Cross Space Mapping via DNN Using Large Scale Click-Through Logs

The gap between low-level visual signals and high-level semantics has been progressively bridged by continuous development of deep neural network (DNN). With recent progress of DNN, almost all image classification tasks have achieved new records of accuracy. To extend the ability of DNN to image retrieval tasks, we proposed a unified DNN model for image-query similarity calculation by simultaneously modeling image and query in one network. The unified DNN is named the cross space mapping (CSM) model, which contains two parts, a convolutional part and a query-embedding part. The image and query are mapped to a common vector space via these two parts respectively, and image-query similarity is naturally defined as an inner product of their mappings in the space. To ensure good generalization ability of the DNN, we learn weights of the DNN from a large number of click-through logs which consists of 23 million clicked image-query pairs between 1 million images and 11.7 million queries. Both the qualitative results and quantitative results on an image retrieval evaluation task with 1000 queries demonstrate the superiority of the proposed method.
Assemble New Object Detector With Few Examples

Learning a satisfactory object detector generally requires sufficient training data to cover the most variations of the object. In this paper, we show that the performance of object detector is severely degraded when training examples are limited. We propose an approach to handle this issue by exploring a set of pretrained auxiliary detectors for other categories. By mining the global and local relationships between the target object category and auxiliary objects, a robust detector can be learned with very few training examples. We adopt the deformable part model proposed by Felzenszwalb and simultaneously explore the root and part filters in the auxiliary object detectors under the guidance of the few training examples from the target object category. An iterative solution is introduced for such a process. The extensive experiments on the PASCAL VOC 2007 challenge data set show the encouraging performance of the new detector assembled from those related auxiliary detectors.
Active tagging for image indexing

Concept labeling and ontology-free tagging are the two typical manners of image annotation. Despite extensive research efforts have been dedicated to labeling, currently automatic image labeling algorithms are still far from satisfactory, and meanwhile manual labeling is rather labor-intensive. In contrast with labeling, tagging works in a free way and therefore it has better user experience for annotators. In this paper, we introduce an active tagging scheme that combines human and computer to assign tags to images. The scheme works in an iterative way. In each round, the most informative images are selected for manual tagging, and the remained images can be annotated by a tag prediction component. We have integrated multiple criteria for sample selection, including ambiguity, citation, and diversity. Experiments are conducted on different datasets and empirical results have demonstrated the effectiveness of the proposed approach.