Miracle-dz/UniFAR-FTUs huggingface dataset:
Exploiting temporal information for 3D pose estimation

In this work, we address the problem of 3D human pose estimation from a sequence of 2D human poses. Although the recent success of deep networks has led many state-ofthe-art methods for 3D pose estimation to train deep networks end-to-end to predict from images directly, the topperforming approaches have shown the effectiveness of dividing the task of 3D pose estimation into two steps: using a state-of-the-art 2D pose estimator to estimate the 2D pose from images and then mapping them into 3D space. They also showed that a low-dimensional representation like 2D locations of a set of joints can be discriminative enough to estimate 3D pose with high accuracy. However, estimation of 3D pose for individual frames leads to temporally incoherent estimates due to independent error in each frame causing jitter. Therefore, in this work we utilize the temporal information across a sequence of 2D joint locations to estimate a sequence of 3D poses. We designed a simple sequence-to-sequence network composed of layernormalized LSTM units with shortcut connections connecting the input to the output on the decoder side and imposed temporal smoothness constraint during training. We found that the knowledge of temporal consistency improves the best reported result by approximately 17.5% and helps our network to recover temporally consistent 3D poses over a sequence of images even when the 2D pose detector fails.
Internet-Scale Data Center Power Efficiency

The Cooperative Expendable Micro-Slice Servers (CEMS) project evaluates low cost, low power servers for high-scale internet-services using commodity, client-side components. It is a follow-on project to the 2007 CIDR paper Architecture for Modular Data Centers [11]. The goals of the CEMS project are to establish that low-cost, low-power servers produce better price/performance and better power/performance than current purpose-built servers. In addition, we aim to establish the viability and efficiency of a fail-in-place model. We use work done per dollar and work done per joule as measures of server efficiency and show that more, lower-power servers produce the same aggregate throughput much more cost effectively and we use measured performance results from a large, consumer internet service to argue this point.
For Your Eyes Only: Learning to Summarize First-Person Videos

With the increasing amount of video data, it is desirable to highlight or summarize the videos of interest for viewing, search, or storage purposes. However, existing summarization approaches are typically trained from third-person videos, which cannot generalize to highlight the first-person ones. By advancing deep learning techniques, we propose a unique network architecture for transferring spatiotemporal information across video domains, which jointly solves metric-learning based feature embedding and keyframe selection via Bidirectional Long Short-Term Memory (BiLSTM). A practical semi-supervised learning setting is considered, i.e., only fully annotated third-person videos, unlabeled first-person videos, and a small amount of annotated first-person ones are required to train our proposed model. Qualitative and quantitative evaluations are performed in our experiments, which confirm that our model performs favorably against baseline and state-of-the-art approaches on first-person video summarization.
Primary Object Segmentation in Aerial Videos via Hierarchical Temporal Slicing and Co-Segmentation

Primary object segmentation plays an important role in understanding videos generated by unmanned aerial vehicles. In this paper, we propose a large-scale dataset APD with 500 aerial videos, in which the primary objects are manually annotated on 5,014 sparsely sampled frames. To the best of our knowledge, it is the largest dataset to date for the task of primary object segmentation in aerial videos. From this dataset, we find that most aerial videos contain large-scale scenes, small sized primary objects as well as consistently varying scales and viewpoints. Inspired by that, we propose a novel hierarchical temporal slicing approach that repeatedly divides a video into two sub-videos formed by the odd and even frames, respectively. In this manner, an aerial video can be represented by a set of hierarchically organized short video clips, and the primary objects they share can be segmented by training end-to-end co-segmentation CNNs and finally refined within the neighborhood reversible flows. Experimental results show that our approach remarkably outperforms 24 state-of-the-art methods in segmenting primary objects in various types of aerial videos.
Sparse Label Smoothing for Semi-supervised Person Re-Identification

Person re-identification (re-id) is a cross-camera retrieval task which establishes a correspondence between images of a person from multiple cameras. Deep Learning methods have been successfully applied to the problem and achieved impressive results. However, these methods require large amounts of labeled training data. Current labeled datasets in person re-id are limited in scale and manually acquiring such large-scale dataset in surveillance camera is a tedious and labor-intensive task. In this paper, we propose a semi-supervised framework that performs Sparse Label Smoothing Regularization (SLSR) by considering similarities between unlabeled sample and training sample in the same feature space. Our approach first exploits the clustering property of existing person re-id datasets to create groups of similar objects that model the correlation among view. We make use of the training set to create cluster of similar objects using the intermediate feature representation of a CNN model. Each cluster is used to generate synthetic data samples using a generative adversarial model. We finally defined a sparse smoothing regularization term and train the network with join supervision of cross-entropy loss. The proposed approach tackles two problems (1) how to efficiently use the generated data and (2) how to address the over-smoothness problem found in current regularization. We solve these two problems by using a generative model for data augmentation and by maintaining and propagating similarities across the network through the concatenation of training images and generated images into one homogeneous feature space. Extensive experiments on four large-scale datasets show that our regularization method significantly improves the Re-ID accuracy compared to existing semi-supervised methods.
An equalised global graphical model-based approach for multi-camera object tracking

Non-overlapping multi-camera visual object tracking typically consists of two steps: single camera object tracking and inter-camera object tracking. Most of tracking methods focus on single camera object tracking, which happens in the same scene, while for real surveillance scenes, inter-camera object tracking is needed and single camera tracking methods can not work effectively. In this paper, we try to improve the overall multi-camera object tracking performance by a global graph model with an improved similarity metric. Our method treats the similarities of single camera tracking and inter-camera tracking differently and obtains the optimization in a global graph model. The results show that our method can work better even in the condition of poor single camera object tracking.
Fully Connected Networks and Generative Neural Networks Applied to Sclera Segmentation

Due to the world’s demand for security systems, biometrics can be seen as an important topic of research in computer vision. One of the biometric forms that has been gaining attention is the recognition based on sclera. The initial and paramount step for performing this type of recognition is the segmentation of the region of interest, i.e. the sclera. In this context, two approaches for such task based on the Fully Connected Network (FCN) and on Generative Adversarial Network (GAN) are introduced in this work. FCN is similar to a common convolution neural network, however the fully connected layers (i.e., the classiﬁcation layers) are removed from the end of the network and the output is generated by combining the output of pooling layers from different convolutional ones. The GAN is based on the game theory, where we have two networks competing with each other to generate the best segmentation. In order to perform fair comparison with baselines and quantitative and objective evaluations of the proposed approaches, we provide to the scientiﬁc community new 1 , 300 manually segmented images from two databases 1 . The experiments are performed on the UBIRIS.v2 and MICHE databases and the best performing conﬁgurations of our propositions achieved F - score (cid:48) s measures of 87 . 48% and 88 . 32% , respectively.
On the Spectral Bias of Deep Neural Networks

It is well known that over-parametrized deep neural networks (DNNs) are an overly expressive class of functions that can memorize even random data with $100\%$ training accuracy. This raises the question why they do not easily overfit real data. To answer this question, we study deep networks using Fourier analysis. We show that deep networks with finite weights (or trained for finite number of steps) are inherently biased towards representing smooth functions over the input space. Specifically, the magnitude of a particular frequency component ($k$) of deep ReLU network function decays at least as fast as $\mathcal{O}(k^{-2})$, with width and depth helping polynomially and exponentially (respectively) in modeling higher frequencies. This shows for instance why DNNs cannot perfectly \textit{memorize} peaky delta-like functions. We also show that DNNs can exploit the geometry of low dimensional data manifolds to approximate complex functions that exist along the manifold with simple functions when seen with respect to the input space. As a consequence, we find that all samples (including adversarial samples) classified by a network to belong to a certain class are connected by a path such that the prediction of the network along that path does not change. Finally we find that DNN parameters corresponding to functions with higher frequency components occupy a smaller volume in the parameter.
Weakly Supervised Local Attention Network for Fine-Grained Visual Classification

In the fine-grained visual classification task, objects usually share similar geometric structure but present different part distribution and variant local features. Therefore, localizing and extracting discriminative local features play a crucial role in obtaining accurate performance. Existing work that first locates specific several object parts and then extracts further local features either require additional location annotation or needs to train multiple independent networks. In this paper. We propose Weakly Supervised Local Attention Network (WS-LAN) to solve the problem, which jointly generates a great many attention maps (region-of-interest maps) to indicate the location of object parts and extract sequential local features by Local Attention Pooling (LAP). Besides, we adopt attention center loss and attention dropout so that each attention map will focus on a unique object part. WS-LAN can be trained end-to-end and achieves the state-of-the-art performance on multiple fine-grained classification datasets, including CUB-200-2011, Stanford Car and FGVC-Aircraft, which demonstrated its effectiveness.
Gated XNOR Networks: Deep Neural Networks with Ternary Weights and Activations under a Unified Discretization Framework

There is a pressing need to build an architecture that could subsume these networks undera unified framework that achieves both higher performance and less overhead. To this end, two fundamental issues are yet to be addressed. The first one is how to implement the back propagation when neuronal activations are discrete. The second one is how to remove the full-precision hidden weights in the training phase to break the bottlenecks of memory/computation consumption. To address the first issue, we present a multistep neuronal activation discretization method and a derivative approximation technique that enable the implementing the back propagation algorithm on discrete DNNs. While for the second issue, we propose a discrete state transition (DST) methodology to constrain the weights in a discrete space without saving the hidden weights. In this way, we build a unified framework that subsumes the binary or ternary networks as its special cases.More particularly, we find that when both the weights and activations become ternary values, the DNNs can be reduced to gated XNOR networks (or sparse binary networks) since only the event of non-zero weight and non-zero activation enables the control gate to start the XNOR logic operations in the original binary networks. This promises the event-driven hardware design for efficient mobile intelligence. We achieve advanced performance compared with state-of-the-art algorithms. Furthermore,the computational sparsity and the number of states in the discrete space can be flexibly modified to make it suitable for various hardware platforms.
Deep Multi-View Clustering via Multiple Embedding

Exploring the information among multiple views usually leads to more promising clustering performance. Most existing multi-view clustering algorithms perform clustering separately: first extracts multiple handcrafted features or deep features, then conducts traditional clustering such as spectral clustering or K-means. However, the learned features may not often work well for clustering. To overcome this problem, we propose the Deep Multi-View Clustering via Multiple Embedding (DMVC-ME), which learns deep embedded features, multi-view fusion mechanism and clustering assignment simultaneously in an end-to-end manner. Specifically, we adopt a KL divergence to refine the soft clustering assignment with the help of a multi-view fused target distribution. The parameters are updated via an efficient alternative optimization scheme. As a result, more clustering-friendly features can be learned and the complementary traits among different views can be well captured. We demonstrate the effectiveness of our approach on several challenging image datasets, where significant superiority can be found over single/multi-view baselines and the state-of-the-art multi-view clustering methods.
Joint Training of Low-Precision Neural Network with Quantization Interval Parameters

Optimization for low-precision neural network is an important technique for deep convolutional neural network models to be deployed to mobile devices. In order to realize convolutional layers with the simple bit-wise operations, both activation and weight parameters need to be quantized with a low bit-precision. In this paper, we propose a novel optimization method for low-precision neural network which trains both activation quantization parameters and the quantized model weights. We parameterize the quantization intervals of the weights and the activations and train the parameters with the full-precision weights by directly minimizing the training loss rather than minimizing the quantization error. Thanks to the joint optimization of quantization parameters and model weights, we obtain the highly accurate low-precision network given a target bitwidth. We demonstrated the effectiveness of our method on two benchmarks: CIFAR-10 and ImageNet.
Inertial-aided Motion Deblurring with Deep Networks

We propose an inertial-aided deblurring method that incorporates gyroscope measurements into a convolutional neural network (CNN). With the help of inertial measurements, it can handle extremely strong and spatially-variant motion blur. At the same time, the image data is used to overcome the limitations of gyro-based blur estimation. To train our network, we also introduce a novel way of generating realistic training data using the gyroscope. The evaluation shows a clear improvement in visual quality over the state-of-the-art while achieving real-time performance. Furthermore, the method is shown to improve the performance of existing feature detectors and descriptors against the motion blur.
A new model to forecast the results of matches based on hybrid neural networks in the soccer rating system

The objective of this paper is to build a result prediction model for the rating system in soccer games. A rating system which plays a crucial role in world sports field yields predictions for the probability that one contestant beats another. The result prediction model is the core technique in the rating system. The robustness and accuracy of the model is a very important feature because people will trust the rating system only if it can give the exact prediction of the game results. This paper employs a coarse-to-fine training technique based on hybrid neural network. Very few people have ever attempted the method based on neural network before in this field. First a match is classified into three categories with a LVQ net to determine the strength contrast between two contestants. Then the elaborately designed data will go through the specific BP nets according to the classifying result. The model is trained and tested on volumes of actual soccer match results from Italian series A. Finally the results of the model are compared to other prediction models based on statistics. The outcome shows that the new model is more accurate and provides better performance evaluation of all teams.
Weakly Supervised Generative Adversarial Networks for 3D Reconstruction

Volumetric 3D reconstruction has witnessed a significant progress in performance through the use of deep neural network based methods that address some of the limitations of traditional reconstruction algorithms. However, this increase in performance requires large scale annotations of 2D/3D data. This paper introduces a novel generative model for volumetric 3D reconstruction, Weakly supervised Generative Adversarial Network (WS-GAN) which reduces reliance on expensive 3D supervision. WS-GAN takes an input image, a sparse set of 2D object masks with respective camera parameters, and an unmatched 3D model as inputs during training. WS-GAN uses a learned encoding as input to a conditional 3D-model generator trained alongside a discriminator, which is constrained to the manifold of realistic 3D shapes. We bridge the representation gap between 2D masks and 3D volumes through a perspective raytrace pooling layer, that enables perspective projection and allows backpropagation. We evaluate WS-GAN on ShapeNet, ObjectNet and Stanford Online Product dataset for reconstruction with single-view and multi-view cases in both synthetic and real images. We compare our method to voxel carving and prior work with full 3D supervision. Additionally, we also demonstrate that the learned feature representation is semantically meaningful through interpolation and manipulation in input space.
Representation Learning and Adversarial Generation of 3D Point Clouds

Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling. In this paper, we look at geometric data represented as point clouds. We introduce a deep autoencoder network for point clouds, which outperforms the state of the art in 3D recognition tasks. We also design GAN architectures to generate novel point-clouds. Importantly, we show that by training the GAN in the latent space learned by the autoencoder, we greatly boost the GAN's data-generating capacity, creating significantly more diverse and realistic geometries, with far simpler architectures. The expressive power of our learned embedding, obtained without human supervision, enables basic shape editing applications via simple algebraic manipulations, such as semantic part editing and shape interpolation.
Is Unit Testing Immune to Coincidental Correctness?

Researchers have previously shown that Coincidental Correctness (CC) is prevalent; however, the benchmarks they used are considered inadequate nowadays. They have also recognized the negative impact of CC on the effectiveness of fault localization and testing. The aim of this paper is to study Coincidental Correctness, using more realistic code, mainly from the perspective of unit testing. This stems from the fact that the practice of unit testing has grown tremendously in recent years due to the wide adoption of software development processes, such as Test-Driven Development. We quantified the presence of CC in unit testing using the Defects4J benchmark. This entailed manually injecting two code checkers for each of the 395 defects in Defects4J: 1) a weak checker that detects weak CC tests by monitoring whether the defect was reached; and 2) a strong checker that detects strong CC tests by monitoring whether the defect was reached and the program has transitioned into an infectious state. We also conducted preliminary experiments (using Defects4J, NanoXML and JTidy) to assess the pervasiveness of CC at the unit testing level in comparison to that at the integration and system levels. Our study showed that unit testing is not immune to CC, as it exhibited 7.2x more strong CC tests than failing tests and 8.3x more weak CC tests than failing tests. However, our preliminary results suggested that it might be less prone to CC than integration testing and system testing.
VSE++: Improved Visual-Semantic Embeddings

This paper investigates the problem of image-caption retrieval using joint visual-semantic embeddings. We introduce a very simple change to the loss function used in the original formulation by Kiros et al. (2014), which leads to drastic improvements in the retrieval performance. In particular, the original paper uses the rank loss which computes the sum of violations across the negative training examples. Instead, we penalize the model according to the hardest negative examples. We then make several additional modifications according to the current best practices in image-caption retrieval. We showcase our model on the MS-COCO and Flickr30K datasets through comparisons and ablation studies. On MS-COCO, we improve caption retrieval by 21% in R@1 with respect to the original formulation. Our results outperform the state-of-the-art results by 8.8% in caption retrieval and 11.3% in image retrieval at R@1. On Flickr30K, we more than double R@1 as reported by Kiros et al. (2014) in both image and caption retrieval, and achieve near state-of-the-art performance. We further show that similar improvements also apply to the Order-embeddings by Vendrov et al. (2015) which builds on a similar loss function.
Re3 : Real-Time Recurrent Regression Networks for Object Tracking

Robust object tracking requires knowledge and understanding of the object being tracked: its appearance, its motion, and how it changes over time. A tracker must be able to modify its underlying model and adapt to new observations. We present Re3, a real-time deep object tracker capable of incorporating long-term temporal information into its model. In line with other recent deep learning techniques, we do not train an online tracker. Instead, we use a recurrent neural network to represent the appearance and motion of the object. We train the network offline to learn how an object's appearance and motion may change, letting it track with a single forward pass at test time. This lightweight model is capable of tracking objects at 150 FPS, while attaining competitive results on challenging benchmarks. We also show that our method handles temporary occlusion better than other comparable trackers using experiments that directly measure performance on sequences with occlusion.