nils-herrmann/clean_scirepeval huggingface dataset:
MMSS: Multi-modal Sharable and Specific Feature Learning for RGB-D Object Recognition

Most of the feature-learning methods for RGB-D object recognition either learn features from color and depth modalities separately, or simply treat RGB-D as undifferentiated four-channel data, which cannot adequately exploit the relationship between different modalities. Motivated by the intuition that different modalities should contain not only some modal-specific patterns but also some shared common patterns, we propose a multi-modal feature learning framework for RGB-D object recognition. We first construct deep CNN layers for color and depth separately, and then connect them with our carefully designed multi-modal layers, which fuse color and depth information by enforcing a common part to be shared by features of different modalities. In this way, we obtain features reflecting shared properties as well as modal-specific properties in different modalities. The information of the multi-modal learning frameworks is back-propagated to the early CNN layers. Experimental results show that our proposed multi-modal feature learning method outperforms state-of-the-art approaches on two widely used RGB-D object benchmark datasets.
Real-Time and Temporal-Coherent Foreground Extraction With Commodity RGBD Camera

Foreground extraction from video stream is an important component in many multimedia applications. By exploiting commodity RGBD cameras, we could further extract dynamic foreground objects with 3D information in real-time, thereby enabling new forms of multimedia applications such as 3D telepresence. However, one critical problem with existing methods for real-time foreground extraction is temporal coherency. They could exhibit severe flickering results for foreground objects such as human motion, thus affecting the visual quality as well as the image object analysis in the multimedia applications. This paper presents a new GPU-based real-time foreground extraction method with several novel techniques. First, we detect shadow and fill missing depth data accordingly in RGBD video, and then adaptively combine color and depth masks to form a trimap. After that, we formulate a novel closed-form matting model to improve the temporal coherency in foreground extraction while achieving real-time performance. Particularly, we propagate RGBD data across temporal domain to improve the visual coherence in the foreground object extraction, and take advantage of various CUDA strategies and spatial data structures to improve the speed. Experiments with a number of users on different scenarios show that, compared with state-of-the-art methods, our method can extract stabler foreground objects with higher visual quality as well as better temporal coherency, while still achieving real-time performance (experimentally, 30.3 frames per second on average).
Adding network-layer intelligence to mobile receivers for solving spurious TCP timeout during vertical handoff

With the help of mobile IP/IPv6 and soft handoff, ongoing TCP sessions can remain active and handoff packet loss can be avoided. However, TCP still faces several performance degradation issues due to the disparities in bandwidth and propagation delay between different access networks. Particularly, during vertical handoffs, some undesirable phenomena may erroneously trigger TCP congestion-control actions and thus degrade TCP performance. In this article we tackle the spurious timeout problem frequently associated with handovers from fast to slow links. We propose three network-layer schemes: fast ACK, slow ACK, and ACK delaying. These schemes require only minor modifications to the network layer of mobile receivers and no change to the TCP protocol and the TCP sender. The simulation results show that these schemes can effectively improve TCP performance during soft vertical handoffs
Physical rate based admission control for HCCA in IEEE 802.11e WLANs

The IEEE 802.11 working group is currently working on the support of quality of service (QoS) in a new standard called IEEE 802.lie which introduces the so-called hybrid coordination function (HCF). A simple admission control unit has already been developed as a reference for HCF controlled channel access (HCCA) in a recent TGe draft. However, this reference scheme is inefficient because it is implemented based on the minimum physical rate from which the mobile stations' actual physical rates deviate greatly at most of the time. In this paper, we propose a physical rate based admission control scheme (PRBAC) which enhances the reference scheme by taking account of both the wireless channel characteristics and the stations' mobility. Numerical analysis and simulation results show the significant improvement of the proposed admission control scheme.
Visual Recognition by Learning From Web Data via Weakly Supervised Domain Generalization

In this paper, a weakly supervised domain generalization (WSDG) method is proposed for real-world visual recognition tasks, in which we train classifiers by using Web data (e.g., Web images and Web videos) with noisy labels. In particular, two challenging problems need to be solved when learning robust classifiers, in which the first issue is to cope with the label noise of training Web data from the source domain, while the second issue is to enhance the generalization capability of learned classifiers to an arbitrary target domain. In order to handle the first problem, the training samples within each category are partitioned into clusters, where we use one bag to denote each cluster and instances to denote the samples in each cluster. Then, we identify a proportion of good training samples in each bag and train robust classifiers by using the good training samples, which leads to a multi-instance learning (MIL) problem. In order to handle the second problem, we assume that the training samples possibly form a set of hidden domains, with each hidden domain associated with a distinctive data distribution. Then, for each category and each hidden latent domain, we propose to learn one classifier by extending our MIL formulation, which leads to our WSDG approach. In the testing stage, our approach can obtain better generalization capability by effectively integrating multiple classifiers from different latent domains in each category. Moreover, our WSDG approach is further extended to utilize additional textual descriptions associated with Web data as privileged information (PI), although testing data do not have such PI. Extensive experiments on three benchmark data sets indicate that our newly proposed methods are effective for real-world visual recognition tasks by learning from Web data.
Retry Limit Based ULP for Scalable Video Transmission over IEEE 802.11e WLANs

We investigate the packet loss behavior in the IEEE 802.11e wireless local area networks (WLANs) under various retry limit settings. Considering scalable video traffic delivery over the IEEE 802.11e WLANs, our study shows the importance of adaptiveness in retry limit settings for the unequal loss protection (ULP) design. Based on the study, we present a simple yet effective retry limit based ULP which adaptively adjusts the retry limit setting of the IEEE 802.11e medium access control protocol to maintain a strong loss protection for critical video traffic transmission. The simulation results illustrate significant advantages in the delivered video quality for our proposed design.
Efficient Video Object Co-Localization With Co-Saliency Activated Tracklets

Video object co-localization is the task of jointly localizing common visual objects across videos. Due to the large variations both across the videos and within each video, it is quite challenging to identify and track the common objects jointly. Unlike the previous joint frameworks that use a large number of bounding box proposals to attack the problem, we propose to leverage co-saliency activated tracklets to efficiently address the problem. To highlight the common object regions, we first explore inter-video commonness, intra-video commonness, and motion saliency to generate the co-saliency maps for a small number of selected key frames at regular intervals. Object proposals of high objectness and co-saliency scores in those frames are tracked across each interval to build tracklets. Finally, the best tube for a video is obtained through selecting the optimal tracklet from each interval with the help of confidence and smoothness constraints. Experimental results on the benchmark YouTube-objects dataset show that the proposed method outperforms the state-of-the-art methods in terms of accuracy and speed under both weakly supervised and unsupervised settings. Moreover, by noticing the existing benchmark dataset lacks of sufficient annotations for object localization (only one annotated frame per video), we further annotate more than 15k frames of the YouTube videos and develop a new benchmark dataset for video co-localization.
Improving TCP performance during soft vertical handoff

With the help of Mobile IP/IPv6, on-going TCP sessions can remain active when the mobile node hands over from one network to another. In particular, in a soft vertical handoff, packet loss due to the switching of networks can be avoided. However, TCP still faces several performance degradation issues due to the change of network bandwidth and propagation delay. During vertical handoff, some undesirable phenomenon may erroneously trigger TCP congestion control operations and thus degrade the TCP performance. In this paper, we study the TCP behaviors during soft vertical handoff from fast link to slow link. In particular, we focus on TCP timeout problem caused by increased network round-trip time, and propose three schemes (fast response, slow response and ACK delaying) to solve this problem. These schemes require only minor modifications on network layer of the end hosts, without any change on TCP and intermediate network routers. Simulation results show that these schemes can effectively improve the TCP performance during soft vertical handoff.
Ubiquitous video streaming through wireless overlay networks

With the availability of various wireless link-layer technologies, such as Bluetooth, WLAN and GPRS, in one wireless device, ubiquitous communications can be realized through managing vertical handoff in the environment of wireless overlay networks. In this paper, we develop a ubiquitous video streaming system through wireless overlay networks. One of the main features of the developed system is that it can support seamless communications. In other words, the developed system enables wireless devices to roam among different wireless networks without affecting the ongoing multimedia applications. Another feature is that our developed video streaming application can be adaptively adjusted to match the significant change of wireless link characteristics due to vertical handoff. Our contribution lies in the integration of various existing techniques to build a real system.
Compact Representation for Image Classification: To Choose or to Compress?

In large scale image classification, features such as Fisher vector or VLAD have achieved state-of-the-art results. However, the combination of large number of examples and high dimensional vectors necessitates dimensionality reduction, in order to reduce its storage and CPU costs to a reasonable range. In spite of the popularity of various feature compression methods, this paper argues that feature selection is a better choice than feature compression. We show that strong multicollinearity among feature dimensions may not exist, which undermines feature compression's effectiveness and renders feature selection a natural choice. We also show that many dimensions are noise and throwing them away is helpful for classification. We propose a supervised mutual information (MI) based importance sorting algorithm to choose features. Combining with 1-bit quantization, MI feature selection has achieved both higher accuracy and less computational cost than feature compression methods such as product quantization and BPBC.
4-D Wavelet-Based Multiview Video Coding

The conventional multiview video coding (MVC) schemes, utilizing both neighboring temporal frames and view frames as possible references, have only shown a slight gain over those using temporal frames alone in terms of coding efficiency. The reason for this is that the neighboring temporal frames exhibit stronger correlation with the current frame and the view frames often fail to be selected as references. This paper proposes an elegant MVC framework using high dimensional wavelet, which rightly matches the inherent high dimension property of multiview video. It also makes a better usage of both temporal and view correlations thanks to the hierarchical decomposition. Besides the proposed framework, this paper also investigates MVC coding from the following aspects. First, a disparity-compensated view filter (DCVF) with pixel alignment is proposed, which can accommodate both global and local view disparities among view frames. The proposed DCVF and the existing motion-compensated temporal filter (MCTF) unify the view and temporal decompositions as a generic lifting transform. Second, an adaptive decomposition structure based on the analysis of the temporal and view correlations is proposed. A Lagrangian cost function is derived to determine the optimum decomposition structure. Third, the major components of the proposed MVC coding are figured out, including macroblock type design, subband coefficient coding, and rate allocation. Extensive experiments are carried out on the MPEG 3DAV test sequences and the superior performance of the proposed MVC coding is demonstrated. In addition, the proposed MVC framework can easily support temporal, spatial, SNR, as well as view scalabilities
Video streaming: an FEC-based novel approach

We present an end-to-end error control scheme based on forward error correction (FEC) for video streaming over the Internet and wireless access networks. We consider various types of transport impairment, including random bit errors, bursty channel error, and packet loss. Because of the stringent delay constraint on video streaming, we believe that FEC is more beneficial than other schemes such as re-transmission to combat transport impairment. In addition to the traditional way of packetizing the compressed video in a orthogonal direction with respect to the channel coding to combat packet loss and bursty errors, we propose a novel scheme of pre-interleaving in order to match the characteristics of video source coding. The application of such pre-interleaving is able to greatly improve the end-to-end performance of video streaming because the adoption of this pre-interleaving can simultaneously satisfy contrasting requirements arising from the channel coding and the source coding. We demonstrate that the proposed pre-interleaving scheme can greatly improve the performance of video streaming over traditional schemes for both Internet and wireless applications.
Modality and Component Aware Feature Fusion for RGB-D Scene Classification

While convolutional neural networks (CNN) have been excellent for object recognition, the greater spatial variability in scene images typically meant that the standard full-image CNN features are suboptimal for scene classification. In this paper, we investigate a framework allowing greater spatial flexibility, in which the Fisher vector (FV) encoded distribution of local CNN features, obtained from a multitude of region proposals per image, is considered instead. The CNN features are computed from an augmented pixel-wise representation comprising multiple modalities of RGB, HHA and surface normals, as extracted from RGB-D data. More significantly, we make two postulates: (1) component sparsity - that only a small variety of region proposals and their corresponding FV GMM components contribute to scene discriminability, and (2) modal non-sparsity - within these discriminative components, all modalities have important contribution. In our framework, these are implemented through regularization terms applying group lasso to GMM components and exclusive group lasso across modalities. By learning and combining regressors for both proposal-based FV features and global CNN features, we were able to achieve state-of-the-art scene classification performance on the SUNRGBD Dataset and NYU Depth Dataset V2.
Automatic image co-segmentation using geometric mean saliency

Most existing high-performance co-segmentation algorithms are usually complicated due to the way of co-labelling a set of images and the requirement to handle quite a few parameters for effective co-segmentation. In this paper, instead of relying on the complex process of co-labelling multiple images, we perform segmentation on individual images but based on a combined saliency map that is obtained by fusing single-image saliency maps of a group of similar images. Particularly, a new multiple image based saliency map extraction, namely geometric mean saliency (GMS) method, is proposed to obtain the global saliency maps. In GMS, we transmit the saliency information among the images using the warping technique. Experiments show that our method is able to outperform state-of-the-art methods on three benchmark co-segmentation datasets.
An Exemplar-Based Multi-View Domain Generalization Framework for Visual Recognition

In this paper, we propose a new exemplar-based multi-view domain generalization (EMVDG) framework for visual recognition by learning robust classifier that are able to generalize well to arbitrary target domain based on the training samples with multiple types of features (i.e., multi-view features). In this framework, we aim to address two issues simultaneously. First, the distribution of training samples (i.e., the source domain) is often considerably different from that of testing samples (i.e., the target domain), so the performance of the classifiers learnt on the source domain may drop significantly on the target domain. Moreover, the testing data are often unseen during the training procedure. Second, when the training data are associated with multi-view features, the recognition performance can be further improved by exploiting the relation among multiple types of features. To address the first issue, considering that it has been shown that fusing multiple SVM classifiers can enhance the domain generalization ability, we build our EMVDG framework upon exemplar SVMs (ESVMs), in which a set of ESVM classifiers are learnt with each one trained based on one positive training sample and all the negative training samples. When the source domain contains multiple latent domains, the learnt ESVM classifiers are expected to be grouped into multiple clusters. To address the second issue, we propose two approaches under the EMVDG framework based on the consensus principle and the complementary principle, respectively. Specifically, we propose an EMVDG_CO method by adding a co-regularizer to enforce the cluster structures of ESVM classifiers on different views to be consistent based on the consensus principle. Inspired by multiple kernel learning, we also propose another EMVDG_MK method by fusing the ESVM classifiers from different views based on the complementary principle. In addition, we further extend our EMVDG framework to exemplar-based multi-view domain adaptation (EMVDA) framework when the unlabeled target domain data are available during the training procedure. The effectiveness of our EMVDG and EMVDA frameworks for visual recognition is clearly demonstrated by comprehensive experiments on three benchmark data sets.
Fast and automatic body circular measurement based on a single kinect

The development of commodity depth camera such as Kinect has paved the way for low-cost human body measurement. The existing Kinect-based body measurement methods are too time-consuming since their purposes also include 3D reconstruction. In this paper, we present a Kinect-based simple but fast and effective solution that is tailored only for human body measurement. In particular, our method captures four views of a subject body and constructs a 3D point cloud for each view. The point clouds are then registered and merged to have a full body cloud on which the body measurements are estimated automatically. The experimental results show that our method requires much shorter execution time than the previous works while still producing measurements with comparable quality.
Improving Videophone Transmission over Multi-Rate IEEE 802.11e Networks

In this paper, we propose an adaptive system for improving videophone transmission over EDCA. We consider that a videophone contains a constant bit rate (CBR) voice source and a rate-adaptive video source. Two issues are addressed in this research. Firstly, how to solve the AP bottleneck problem, and secondly, how to adjust video source rate to improve the network performance. For the first issue, we propose the adjustment of the transmission opportunity (TXOP) to give AP a higher priority in voice transmission in order to eliminate the AC3 transmission bottleneck at the AP. For the second issue, our principle is to guarantee the throughput of voice traffic while transmitting as much video traffic as possible. Moreover, we consider more realistic multi-rate WLANs, where multiple transmission rates are used in the PHY layer depending on the underlaying channel conditions.
An On-Off Queue Control Mechanism for Scalable Video Streaming over the IEEE 802.11e WLAN

In this paper, we study the issue of scalable video streaming over IEEE 802.11e EDCA WLANs. Our basic idea is to control the number of "active" nodes on the channel in order to reduce collisions under heavy traffic conditions. Specifically, we propose a distributed on-off queue control (OOQC) mechanism, which is designed to maintain high network throughput while keeping packet loss due to collision as low as possible. A low priority early drop (LPED) method is also employed to drop the packets at the queue according to packet relative priority index (RPI) provided by scalable video coding. Simulation results show that our proposed OOQC scheme significantly outperforms EDCA in received video quality.
Towards Efficient Resource Allocation for Heterogeneous Workloads in IaaS Clouds

Infrastructure-as-a-service (IaaS) cloud technology has attracted much attention from users who have demands on large amounts of computing resources. Current IaaS clouds provision resources in terms of virtual machines (VMs) with homogeneous resource configurations where different types of resources in VMs have similar share of the capacity in a physical machine (PM). However, most user jobs demand different amounts for different resources. For instance,high-performance-computing jobs require more CPU cores while big data processing applications require more memory. The existing homogeneous resource allocation mechanisms cause resource starvation where dominant resources are starved while non-dominant resources are wasted. To overcome this issue, we propose a heterogeneous resource allocation approach, called skewness-avoidance multi-resource allocation (SAMR), to allocate resource according to diversified requirements on different types of resources. Our solution includes a VM allocation algorithm to ensure heterogeneous workloads are allocated appropriately to avoid skewed resource utilization in PMs, and a model-based approach to estimate the appropriate number of active PMs to operate SAMR. We show relatively low complexity for our model-based approach for practical operation and accurate estimation. Extensive simulation results show the effectiveness of SAMR and the performance advantages over its counterparts.