nils-herrmann/clean_scirepeval huggingface dataset:
Convolutional neural networks at constrained time cost

Though recent advanced convolutional neural networks (CNNs) have been improving the image recognition accuracy, the models are getting more complex and time-consuming. For real-world applications in industrial and commercial scenarios, engineers and developers are often faced with the requirement of constrained time budget. In this paper, we investigate the accuracy of CNNs under constrained time cost. Under this constraint, the designs of the network architectures should exhibit as trade-offs among the factors like depth, numbers of filters, filter sizes, etc. With a series of controlled comparisons, we progressively modify a baseline model while preserving its time complexity. This is also helpful for understanding the importance of the factors in network designs. We present an architecture that achieves very competitive accuracy in the ImageNet dataset (11.8% top-5 error, 10-view test), yet is 20% faster than “AlexNet”  (16.0% top-5 error, 10-view test).
Computing nearest-neighbor fields via Propagation-Assisted KD-Trees

Matching patches between two images, also known as computing nearest-neighbor fields, has been proven a useful technique in various computer vision/graphics algorithms. But this is a computationally challenging nearest-neighbor search task, because both the query set and the candidate set are of image size. In this paper, we propose Propagation-Assisted KD-Trees to quickly compute an approximate solution. We develop a novel propagation search method for kd-trees. In this method the tree nodes checked by each query are propagated from the nearby queries. This method not only avoids the time-consuming backtracking in traditional tree methods, but is more accurate. Experiments on public data show that our method is 10-20 times faster than the PatchMatch method  at the same accuracy, or reduces its error by 70% at the same running time. Our method is also 2-5 times faster and is more accurate than Coherency Sensitive Hashing [22], a latest state-of-the-art method.
Accelerating Very Deep Convolutional Networks for Classification and Detection

This paper aims to accelerate the test-time computation of convolutional neural networks (CNNs), especially very deep CNNs <xref ref-type="bibr" rid="ref1">[1]</xref> that have substantially impacted the computer vision community. Unlike previous methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into account. We develop an effective solution to the resulting nonlinear optimization problem without the need of stochastic gradient descent (SGD). More importantly, while previous methods mainly focus on optimizing one or two layers, our nonlinear method enables an asymmetric reconstruction that reduces the rapidly accumulated error when multiple (e.g., <inline-formula> <tex-math notation="LaTeX">$\ge$</tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq1-2502579.gif"/> </alternatives></inline-formula>10) layers are approximated. For the widely used very deep VGG-16 model <xref ref-type="bibr" rid="ref1">[1]</xref> , our method achieves a whole-model speedup of 4<inline-formula><tex-math notation="LaTeX">$\times$</tex-math> <alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq2-2502579.gif"/></alternatives></inline-formula> with merely a 0.3 percent increase of top-5 error in ImageNet classification. Our 4<inline-formula><tex-math notation="LaTeX">$\times$ </tex-math><alternatives><inline-graphic xlink:type="simple" xlink:href="he-ieq3-2502579.gif"/></alternatives></inline-formula> accelerated VGG-16 model also shows a graceful accuracy degradation for object detection when plugged into the Fast R-CNN detector <xref ref-type="bibr" rid="ref2">[2]</xref> .
Aggregated Residual Transformations for Deep Neural Networks

We present a simple, highly modularized network architecture for image classification. Our network is constructed by repeating a building block that aggregates a set of transformations with the same topology. Our simple design results in a homogeneous, multi-branch architecture that has only a few hyper-parameters to set. This strategy exposes a new dimension, which we call cardinality (the size of the set of transformations), as an essential factor in addition to the dimensions of depth and width. On the ImageNet-1K dataset, we empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy. Moreover, increasing cardinality is more effective than going deeper or wider when we increase the capacity. Our models, named ResNeXt, are the foundations of our entry to the ILSVRC 2016 classification task in which we secured 2nd place. We further investigate ResNeXt on an ImageNet-5K set and the COCO detection set, also showing better results than its ResNet counterpart. The code and models are publicly available online.
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224 × 224) input image. This requirement is "artificial" and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, "spatial pyramid pooling", to eliminate the above requirement. The new network structure, called SPP-net, can generate a fixed-length representation regardless of image size/scale. Pyramid pooling is also robust to object deformations. With these advantages, SPP-net should in general improve all CNN-based image classification methods. On the ImageNet 2012 dataset, we demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures despite their different designs. On the Pascal VOC 2007 and Caltech101 datasets, SPP-net achieves state-of-the-art classification results using a single full-image representation and no fine-tuning. The power of SPP-net is also significant in object detection. Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training the detectors. This method avoids repeatedly computing the convolutional features. In processing test images, our method is 24-102 × faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007. In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our methods rank #2 in object detection and #3 in image classification among all 38 teams. This manuscript also introduces the improvement made for this competition.
Transitive Invariance for Self-Supervised Visual Representation Learning

Learning visual representations with self-supervised learning has become popular in computer vision. The idea is to design auxiliary tasks where labels are free to obtain. Most of these tasks end up providing data to learn specific kinds of invariance useful for recognition. In this paper, we propose to exploit different self-supervised approaches to learn representations invariant to (i) inter-instance variations (two objects in the same class should have similar features) and (ii) intra-instance variations (viewpoint, pose, deformations, illumination, etc.). Instead of combining two approaches with multi-task learning, we argue to organize and reason the data with multiple variations. Specifically, we propose to generate a graph with millions of objects mined from hundreds of thousands of videos. The objects are connected by two types of edges which correspond to two types of invariance: “different instances but a similar viewpoint and category” and “different viewpoints of the same instance”. By applying simple transitivity on the graph with these edges, we can obtain pairs of images exhibiting richer visual invariance. We use this data to train a Triplet-Siamese network with VGG16 as the base architecture and apply the learned representations to different recognition tasks. For object detection, we achieve 63.2% mAP on PASCAL VOC 2007 using Fast R-CNN (compare to 67.3% with ImageNet pre-training). For the challenging COCO dataset, our method is surprisingly close (23.5%) to the ImageNet-supervised counterpart (24.4%) using the Faster R-CNN framework. We also show that our network can perform significantly better than the ImageNet network in the surface normal estimation task.
K-Means Hashing: An Affinity-Preserving Quantization Method for Learning Binary Compact Codes

In computer vision there has been increasing interest in learning hashing codes whose Hamming distance approximates the data similarity. The hashing functions play roles in both quantizing the vector space and generating similarity-preserving codes. Most existing hashing methods use hyper-planes (or kernelized hyper-planes) to quantize and encode. In this paper, we present a hashing method adopting the k-means quantization. We propose a novel Affinity-Preserving K-means algorithm which simultaneously performs k-means clustering and learns the binary indices of the quantized cells. The distance between the cells is approximated by the Hamming distance of the cell indices. We further generalize our algorithm to a product space for learning longer codes. Experiments show our method, named as K-means Hashing (KMH), outperforms various state-of-the-art hashing encoding methods.
Constant Time Weighted Median Filtering for Stereo Matching and Beyond

Despite the continuous advances in local stereo matching for years, most efforts are on developing robust cost computation and aggregation methods. Little attention has been seriously paid to the disparity refinement. In this work, we study weighted median filtering for disparity refinement. We discover that with this refinement, even the simple box filter aggregation achieves comparable accuracy with various sophisticated aggregation methods (with the same refinement). This is due to the nice weighted median filtering properties of removing outlier error while respecting edges/structures. This reveals that the previously overlooked refinement can be at least as crucial as aggregation. We also develop the first constant time algorithm for the previously time-consuming weighted median filter. This makes the simple combination ``box aggregation + weighted median'' an attractive solution in practice for both speed and accuracy. As a byproduct, the fast weighted median filtering unleashes its potential in other applications that were hampered by high complexities. We show its superiority in various applications such as depth up sampling, clip-art JPEG artifact removal, and image stylization.
Efficient and accurate approximations of nonlinear convolutional networks

This paper aims to accelerate the test-time computation of deep convolutional neural networks (CNNs). Unlike existing methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into account. We minimize the reconstruction error of the nonlinear responses, subject to a low-rank constraint which helps to reduce the complexity of filters. We develop an effective solution to this constrained nonlinear optimization problem. An algorithm is also presented for reducing the accumulated error when multiple layers are approximated. A whole-model speedup ratio of 4× is demonstrated on a large network trained for ImageNet, while the top-5 error rate is only increased by 0.9%. Our accelerated model has a comparably fast speed as the “AlexNet” [11], but is 4.7% more accurate.
Deep Residual Learning for Image Recognition

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
Complexity-driven rate-control for parallel HEVC coding

When using adaptive streaming, the content needs to be segmented so that clients can seamlessly switch to different rates depending on network conditions. On the video server each segment is stored in various bitrate representations, which are in practice provided by very fast encoders. Such encoders rely on parallelization strategies to limit the encoder complexity. Parallelization strongly affects the performance of rate-control (RC) algorithms, since different segments and parts of segments are encoded independently from each other. A new approach is proposed in this paper to tackle these issues, based on the optimization of the initial parameters of a state-of-the-art RC model for inter-predicted frames in an HEVC/H.265 codec. The model makes use of an estimate of the texture complexity of the first frame in the segment to efficiently tune the parameters depending on the target rate. The approach is consistently improving the accuracy of RC schemes as well as the visual quality, with negligible impact on the encoding efficiency.
Robust Region-of-Interest Scalable Coding with Leaky Prediction in H.264/AVC

We propose a region-of-interest (ROI) coding framework based on leaky prediction (LP) for robustly transporting H.264 video over error-prone network. The LP-based ROI coding can remove error drift caused by the destruction in the decoded background from the proposed system, with guaranteed ROI quality throughout. Experimental results show that this scheme enables the decoder to reconstruct the ROI with better quality and the global video frame with improved quality even if the background bitstream cannot be correctly received or completely lost.
GOP-Level Transmission Distortion Modeling for Unequal Importance Judgement

In the case of transmitting stored video streaming over packet-switched networks, unavoidable frame losses may result in error propagation of reconstructed video and thus induce severe quality degradation. In order to minimize the transmission distortion and utilize the limited channel resources efficiently, unequal error protection is usually adopted to exploit the unequal importance of different frames in one group-of-pictures (GOP). In this paper, we develop an estimate model for GOP-level transmission distortion so that the transmitter is able to find the unequal importance of different frames in one GOP. The simulation results demonstrate that the proposed model is accurate and robust.
Image classification based on pyramid histogram of topics

In this paper we propose PHOTO (Pyramid Histogram Of TOpics), a new representation for image classification. We partition the image into hierarchical cells and learn the topic histogram using pLSA over each cell with EM algorithm. Then we concatenate the topic histograms over the cells at all levels to form a “long” vector, i.e. pyramid histogram of topics. Finally AdaBoost classifiers are used to select the topics most discriminative for class recognition. Experimental results on two diverse databases show that our method performs significantly better than general topic representation.
An image quality assessment metric based on quaternion wavelet transform

In recent years, a great deal of efforts has been made to develop objective image quality assessment (IQA) that shows consistency with subjective quality evaluation. After structure similarity was found to be an important factor, IQA metrics such as SSIM showed a better performance than traditional ones. With the usage of phase congruency (PC) to obtain structure similarity measurement, a new metric named FSIM acquired even higher consistency. This paper is motivated to propose an improved structure similarity metric based on FSIM, where quaternion wavelet transform is exploited to extract the quaternion phase congruency map to represent the essential image structures. In addition, complex phase congruency map obtained as the co-product during quaternion wavelet transform, supplies complementary visual effects of detailed structure on image quality assessment. The proposed image quality assessment metric based on hybrid phase congruency (HPC) (IQA_HPC) is highlighted in emphasizing saliency distribution of different image structures to image quality assessment. As compared with the state-of-the art IQA methods, experimental results demonstrate that the new index IQA_HPC is able to gain a higher consistency with the subjective measurement.
Motion Vector Smoothing for True Motion Estimation

This paper proposes a new motion vector (MV) smoothing algorithm to track the real motion in image sequences for MPEG video encoders. First, a pre-checking algorithm is employed to eliminate wrong motion vectors and preserve all possible motion vectors. For each block considered, the motion similarity between the neighboring blocks and the number of candidate motion vectors are jointly exploited to adaptively grow the filtering support, which is supposed to have homogeneous motion and sufficient spatial gradient. Then, all candidate motion vectors are checked within the filtering support using a new motion smoothness-constrained matching criteria. The simulation results show that the proposed algorithm can efficiently track the real motion resulting in smooth motion vector field (MVF)
A Hierarchical Diffusion Algorithm for Community Detection in Social Networks

Community discovery is one of the most important steps to understand the social networks. We propose a hierarchical diffusion method to detect the community structure. Our algorithm is based on the idea that people in different communities usually share less common friends. We also make use of the fact that people usually make decisions based others’choices, especially their friends’. Our algorithm can distinguish between pseudo-communities and meaningful ones. Tests on both classical and synthetic benchmarks show that our algorithm is comparable to state-of-the-art community detection algorithms in both computational complexity and accuracy measured by the so-called normalized mutual information.
Color transfer via local binary patterns mapping

Color transfer is a process of carrying over image colors from one image to another. Since images have diverse texture, color, content and other features, key challenge for color transfer is to find a correct mapping between image and target image. In this paper, a new color transfer method based on feature points extraction and local binary patterns(LBP) mapping is proposed. We construct a framework for feature points extraction and mapping. In the framework, we use SIFT descriptor for feature points selection and we design a LBP mapping method for feature points matching between source and target image. Final feature points in source image are obtained using this mapping. Greyscale source image and final feature points are used as inputs of a scribble-based colorization method to get the final color transfer result. Experimental results show the benefits of our method.
Learning sparse dictionaries with a popularity-based model

Sparse signal representation based on overcomplete dictionaries has recently been extensively investigated, rendering the state-of-the-art results in signal, image and video processing. We propose a novel dictionary learning algorithm—the PK-SVD algorithm—which assumes prior probabilities on the dictionary atoms and learns a sparse dictionary under a popularity-based model. The prior distribution brings the flexibility that is desirable in applications. We examine our algorithm in both synthetic tests and image denoising experiments.
Development of frequency step tunable 1 MW gyrotrons in D-band

Gyrotrons are widely used in electron-cyclotron-plasma heating of fusion installations. Modern and future installations require microwave sources with power of at least 1 MW. There are two main directions in the development of megawatt gyrotrons at GYCOM. The first line is an enhancement of parameters for conventional gyrotrons. The main efforts are applied now to implementing of CVD diamond windows into the gyrotrons and providing gyrotron operation in CW (or very long pulses of tens seconds) regime.
Online Pedestrian Tracking Using Ensemble Color Feature

In this paper, we propose a novel online visual tracking algorithm using ensemble color feature. Visual tracking has long been one of the most important topics in computer vision. In recent years, significant progress has been achieved by the state-of-the-art tracking algorithms. However, some challenging problems still remain unsolved. One general problem is that it is difficult for a tracking algorithm to achieve both high accuracy and efficiency. More specifically, most tracking algorithms base on the hypothesis that the tracking object is a rigid body. Thus, some visual features are disabled when the hypothesis is invalid. Meanwhile, the existing trackers tend to fail when the target object changes poses, because such changes lead to the dramatic mismatch between the momentaneous appearance model and the learned one. The last problem is especially serious when the tracking targets are pedestrians, which suffer irregular pose and shape changes while they walk or run. The proposed algorithm applies ensemble color feature model. It takes pedestrians' physiological structure into consideration, and takes advantage of color histogram information under several color spaces, including the RGB, normRGB, HSV, and Lab. Since histogram is a statistic-based feature, and the garments of pedestrians provide abundant color information, our appearance model using ensemble color feature is more robust to pose and shape changes. Experiments on Caltech Pedestrian Database with evaluation of the state-of-the-art algorithms show better performance of the proposed algorithm.
Online trendy topics detection in microblogs with selective user monitoring under cost constraints

As microblog services such as Twitter become a fast and convenient communication approach, identification of trendy topics in microblog services has great academic and business value. However detecting trendy topics is very challenging due to huge number of users and short-text posts in microblog diffusion networks. In this paper we introduce a trendy topics detection system under computation and communication resource constraints. In stark contrast to retrieving and processing the whole microblog contents, we develop an idea of selecting a small set of microblog users and processing their posts to achieve an overall acceptable trendy topic coverage, without exceeding resource budget for detection. We formulate the selection operation of these subset users as mixed-integer optimization problems, and develop heuristic algorithms to compute their approximate solutions. The proposed system is evaluated with real-time test data retrieved from Sina Weibo, the dominant microblog service provider in China. It's shown that by monitoring 500 out of 1.6 million microblog users and tracking their microposts (about 15,000 daily) with our system, nearly 65% trendy topics can be detected, while on average 5 hours earlier before they appear in Sina Weibo official trends.