nils-herrmann/clean_scirepeval huggingface dataset:
CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

We present variational generative adversarial networks, a general learning framework that combines a variational auto-encoder with a generative adversarial network, for synthesizing images in fine-grained categories, such as faces of a specific person or objects in a category. Our approach models an image as a composition of label and latent attributes in a probabilistic model. By varying the fine-grained category label fed into the resulting generative model, we can generate images in a specific category with randomly drawn values on a latent attribute vector. Our approach has two novel aspects. First, we adopt a cross entropy loss for the discriminative and classifier network, but a mean discrepancy objective for the generative network. This kind of asymmetric loss function makes the GAN training more stable. Second, we adopt an encoder network to learn the relationship between the latent space and the real image space, and use pairwise feature matching to keep the structure of generated images. We experiment with natural images of faces, flowers, and birds, and demonstrate that the proposed models are capable of generating realistic and diverse samples with fine-grained category labels. We further show that our models can be applied to other tasks, such as image inpainting, super-resolution, and data augmentation for training better face recognition models.
Nuclei Segmentation Using Marker-Controlled Watershed, Tracking Using Mean-Shift, and Kalman Filter in Time-Lapse Microscopy

It is important to observe and study cancer cells' cycle progression in order to better understand drug effects on cancer cells. Time-lapse microscopy imaging serves as an important method to measure the cycle progression of individual cells in a large population. Since manual analysis is unreasonably time consuming for the large volumes of time-lapse image data, automated image analysis is proposed. Existing approaches dealing with time-lapse image data are rather limited and often give inaccurate analysis results, especially in segmenting and tracking individual cells in a cell population. In this paper, we present a new approach to segment and track cell nuclei in time-lapse fluorescence image sequence. First, we propose a novel marker-controlled watershed based on mathematical morphology, which can effectively segment clustered cells with less oversegmentation. To further segment undersegmented cells or to merge oversegmented cells, context information among neighboring frames is employed, which is proved to be an effective strategy. Then, we design a tracking method based on modified mean shift algorithm, in which several kernels with adaptive scale, shape, and direction are designed. Finally, we combine mean-shift and Kalman filter to achieve a more robust cell nuclei tracking method than existing ones. Experimental results show that our method can obtain 98.8% segmentation accuracy, 97.4% cell division tracking accuracy, and 97.6% cell tracking accuracy
Low-complexity asymmetric multiview video coding

Multiview video coding (MVC) is currently under development by the Joint Video Team (JVT) as an extension to Advanced Video Coding (H264/AVC). Based on the suppression theory in binocular vision, the fidelity of one of the two views of a stereoscopic display can be reduced without noticeable degradation of subjective quality. Thus, in MVC, a subset of views can be coded with lower spatial resolution at negligible cost to subjective quality. Due to different resolutions, a downsampling process is required in an MVC decoder in order to enable motion compensation (MC) between views. In this paper, a low-complexity MC algorithm is proposed for MVC to enable inter-view prediction between pictures with different resolutions. It requires lower memory consumption and lower computational complexity compared with the conventional downsampled inter-view prediction, while providing comparable efficiency, as shown by the simulation results.
Smoothing rate control for multiple video streams using game theory

To fairly allocate network bandwidth for different streams and to achieve a smooth quality between different video sequences are two key issues in multi-stream applications. In this paper, a new joint rate control scheme for multiple video streams based on game theory is proposed. We achieve a fair rate allocation in video quality between different sequences by Nash Bargaining theory under the constraints of buffer and bandwidth. Also, by grouping all the different sequences' frames within the same timeslot into a coalition, we can avoid the fluctuation in a sequence by bargaining between current coalition and remaining coalitions. Some experiments have been conducted to verify the proposed method.
Coding of mixed-resolution multiview video in 3D video application

The emerging MVC+D standard specifies the coding of Multiview Video plus Depth (MVD) data for enabling advanced 3D video applications. MVC+D specifications define the coding of all views of MVD at equal spatial resolution and apply a conventional MVC technique for coding the multiview texture and the depth independently. This paper presents a modified MVC+D coding scheme, where only the base view is coded at the original resolution whereas dependent views are coded at reduced resolution. To enable inter-view prediction, the base view is downsampled within the MVC coding loop to provide a relevant reference for dependent views. At the decoder side, the proposed scheme consists of a post-processing scheme which upsamples of the decoded views to their original resolution. The proposed scheme is compared against the original MVC+D scheme and an average of 4% delta bitrate reduction (dBR) in the coded views and 14.5% of dBR in the synthesized views are reported.
Refining QP to improve coding efficiency in AVS

This paper presents a Lagrange multiplier determination method and Quantization Parameter (QP) refinement algorithm used in the Rate-Distortion Optimization (RDO) process to improve the coding efficiency for Audio Video Coding Standard (AVS) (IEEE 1857). This paper investigates the Lagrange multiplier setting problem for different kinds of pictures in the encoding process of AVS. When Lagrange multiplier is determined, to minimize the RD (Rate-Distortion) cost, the optimal QP can be selected by multiple-QP optimization. However, this kind of optimization increases the encoding time significantly. This paper analyzes the relationship between the QP value and Lagrange multiplier in this paper. And then this relationship is applied in the encoding process of AVS to refine the predetermined QP value. The experimental results show that proposed algorithm can lead to significant bit saving compared with the default anchor of AVS. The proposed method improves the coding efficiency without increasing the processing time.
Error Resilient Video Coding using Redundant Pictures

Coding of redundant pictures is supported in the latest international video coding standard H.264 (also known as MPEG-4 part 10 or AVC). This paper proposes a standard-compliant way to encode and decode redundant pictures for improved error resilience. The method is based on a combination of picture-level reference picture selection, reference picture list ordering, and a hierarchical allocation of redundant pictures, which can efficiently prevent temporal error propagation without relying on feedback information. Simulation results show that the method outperforms the optimal loss-aware rate-distortion optimized intra refresh method. The proposed algorithm has been adopted into the H.264 joint model.
$\lambda $  Domain Rate Control Algorithm for High Efficiency Video Coding

Rate control is a useful tool for video coding, especially in real-time communication applications. Most of existing rate control algorithms are based on the R - Q model, which characterizes the relationship between bitrate R and quantization Q, under the assumption that Q is the critical factor on rate control. However, with the video coding schemes becoming more and more flexible, it is very difficult to accurately model the R - Q relationship. In fact, we find that there exists a more robust correspondence between R and the Lagrange multiplier λ. Therefore, in this paper, we propose a novel λ-domain rate control algorithm based on the R - λ model, and implement it in the newest video coding standard high efficiency video coding (HEVC). Experimental results show that the proposed λ-domain rate control can achieve the target bitrates more accurately than the original rate control algorithm in the HEVC reference software as well as obtain significant R-D performance gain. Thanks to the high accurate rate control algorithm, hierarchical bit allocation can be enabled in the implemented video coding scheme, which can bring additional R-D performance gain. Experimental results demonstrate that the proposed λ-domain rate control algorithm is effective for HEVC, which outperforms the R - Q model based rate control in HM-8.0 (HEVC reference software) by 0.55 dB on average and up to 1.81 dB for low delay coding structure, and 1.08 dB on average and up to 3.77 dB for random access coding structure. The proposed λ-domain rate control algorithm has already been adopted by Joint Collaborative Team on Video Coding and integrated into the HEVC reference software.
Region based motion vector prediction using data hiding and decoder side reasoning

Motion information occupies a considerable portion of the bit stream generated by the state-of-the-art video codecs. As a sequence, reducing bits on motion information can improve the overall performance of video compression. Various competition based motion vector coding (MVC) schemes were proposed for this goal. In this paper we propose a region based motion vector prediction (RMVP) algorithm to save signaling bits in conventional MVC schemes. Specifically, the motion vector predictor selection information is embedded into the parity of the motion vector differences which would be transmitted into the decoder. To obtain the modulated motion vector differences, the encoder needs to divide the reference frame into several regions and then perform constrained motion estimation in some of the regions. To confirm the advantages of RMVP over different MVC schemes, we implemented it on both KTA Software and TMuC test model. Experimental results show that RMVP outperforms existing MVC schemes.
Two-stage hough transform algorithm for lane detection system based on TMS320DM6437

This paper presents a novel two-stage Hough transform (HT) algorithm and how to use it on a lane detection system for vehicles and industrial robot based on TMS320DM6437. As is known to all, white and yellow are the basic color of lane mark, which can be easily extracted through YCbCr color space. In order to improve detection performance, the two-stage HT is proposed to act on a binary image which is produced by setting threshold for ‘Y’ component and ‘Cb’ component. As lane detection algorithm usually does, HT is used on a edge image first. Then the two-stage Hough transform use the same accumulator with the first time. Finally, an Enhanced Hough Transform Graph (EHTG) is generated. The EHTG contains the information both edge and color. Thus, we can utilize it to visualize lane and transplant the algorithm on DSP. After a series of optimization in programming, the result shows that the proposed algorithm give an outstanding performance on DM6437 platform despite of lane mark embedded in severe noise environment.
Intermediate description for multiple video adaptation

Video adaptation has been considered as a promising technique to tackle challenging problems in pervasive media applications. However, the style of video representation and description in current framework is too simple to accommodate diversified usages, especially for realtime application environment. In this paper, an intermediate video description termed intermedia is proposed, which is generated by off-line pre-processing the original video signal and includes abundant information of video content. On the basis of intermedia, a novel framework for video adaptation can be implemented to handle multiple video applications. The framework has low complexity operations not only in signal level, but also in structural level, and in joint level. Texture, motion and structural characteristic are included in intermedia, so that the framework can tackle a number of different adaptation applications, e.g., bit-rate conversion, temporal resolution reduction, video summarization browsing, and multi-dimensional adaptation. Experimental results demonstrate that intermedia can support multiple video adaptations rapidly while maintaining coding efficiency.
Adaptive Redundant Picture for Error Resilient Video Coding

We present several efficient adaptive redundant picture coding methods for error resilient video coding. In our previous work, redundant picture coding in combining with reference picture selection, reference picture list reordering and hierarchical redundant picture allocation was proposed. This paper investigates how to allocate redundant pictures more efficiently according to the content characteristics of the primary pictures. Simulation results show that the adaptive redundant picture coding methods can achieve average PSNR improvements around 2 to 4 dB compared to the loss-aware rate distortion optimized (LA-RDO) intra macroblock refresh implemented in H.264/AVC Joint Model (JM). This paper also asserts that the methods do not introduce any additional end-to-end delay, therefore suit for low-delay applications such as video telephony and video conferencing, which demand better error resilience than other applications.
Distributed residual coding for multi-view video with joint motion vector projection and 3-D warping

Conventional multi-view video coding (MVC) requires high encoding complexity and high rate data transmission from cameras to a joint encoder. Distributed coding theories guarantee that low complexity and independent encoders can achieve similar coding efficiency as traditional methods. However, there is still a big performance gap between current distributed multi-view video coding (DMVC) schemes and conventional schemes. In this paper, we propose a novel distributed residual coding framework for multi-view video with joint motion vector projection (MVP) and 3-D warping. At the encoder, by means of the improved block classification, distributed residual coding is performed effectively to help reduce temporal redundancy. At the decoder, a joint motion vector projection and 3-D warping method is proposed to generate the candidates of side information and two binary fusion masks. Finally, we employ a novel fusion criterion with relation to the reliability of side information acquisition measures. Experimental results show that the proposed DMVC framework has achieved both side information quality and rate distortion performance improvement.
A comparison between SVC and transcoding

This paper presents a comparison between the scalable extension of H.264/AVC (SVC) and transcoding, in terms of rate-distortion (RD) performance and computational complexity. Both bitrate and spatial resolution reduction cases are included. Simulation results show that, in the aspect of RD performance, SVC performs significantly better in bitrate reduction and slightly better in spatial resolution reduction. Meanwhile, SVC adaptation through bitstream extraction has negligible complexity compared with transcoding.
Distortion Bounds for Source Broadcast Problems

This paper investigates the joint source-channel coding problem of sending a memoryless source over a memoryless broadcast channel. An inner bound and several outer bounds on the admissible distortion region are derived, which, respectively, generalize and unify several existing bounds. As a consequence, we also obtain an inner bound and an outer bound for the degraded broadcast channel case. When specialized to the Gaussian or binary source broadcast, the inner bound and outer bound not only recover the best known inner bound and outer bound in the literature but also generate some new results. Besides, we also extend the inner bound and outer bounds to the Wyner–Ziv source broadcast problem, i.e., source broadcast with side information available at decoders. Some new bounds are obtained when specialized to the Wyner–Ziv Gaussian and Wyner–Ziv binary cases.
Sign language recognition based on adaptive HMMS with data augmentation

Vision based sign language recognition (SLR) is a challenging task due to the complexity of signs and limited data collection. To improve the recognition precision, this paper proposes an adaptive GMM-based (Gaussian mixture model) HMMs (Hidden Markov Models) framework. We discover that inherent latent states in HMMs are not only related to the number of key gestures and body poses, but also related to the kinds of their translation relationships. We propose adaptive HMMs and obtain the hidden state number for each sign with affinity propagation clustering. Furthermore, to enrich the training dataset, we propose a data augmentation strategy by adding Gaussian random disturbances. Experiments on a vocabulary of 370 signs demonstrate the effectiveness of our proposed method over the comparison algorithms.
Wireless Cooperative Video Coding Using a Hybrid Digital–Analog Scheme

Wireless video broadcast/multicast and mobile video communication pose a challenge to the conventional video transmission strategy (which applies separate digital source coding and digital channel coding in a point-to-point communication system) and, therefore, cooperative communication has been proposed to improve the received quality of the receivers with bad channels and the robustness to fading (mobility). In this paper we propose a novel wireless cooperative video coding (WCVC) framework. Specifically, we present a cooperative joint source-channel coding scheme that is based on hybrid digital-analog coding and integrates the advantages of digital coding and analog coding. Compared with most state-of-the-art cooperative video delivery methods, no matter for cooperation scenario or noncooperation scenario, the proposed scheme can avoid the staircase effect and realize continuous quality scalability on condition that the channel quality is within the expected range, and it has strong adaptability to channel variation with higher coding efficiency and better fairness among all receivers. Therefore, it is very suitable for wireless cooperative/noncooperative video broadcast/multicast transmission and cooperative/noncooperative mobile video applications. The experimental results show that the proposed WCVC outperforms SoftCast (which is a new analog scheme) and SVC + hierarchical modulation (HM) (which combines H.264/SVC codec and HM technique) no matter the noncooperative scenario or for cooperative scenario, which verify the effectiveness of our proposed WCVC framework.
Automated segmentation and tracking of cells in time-lapse microscopy using watershed and mean shift

In this paper, we present a new method combining watershed and mean shift for segmentation and tracking of cancer cell nuclei in time-lapse fluorescence. First, we apply the watershed algorithm to segment the cells in each frame of the video sequence, including clustered cells. Second, mean shift method is employed to track each cell in its cycle progression. The proposed method can automatically segment and track all cells without any manual initialization. Experimental result shows that our method can detect almost all the touching cells and track them successfully, especially in the case of cell mitosis which is a difficult task using traditional methods such as snake and level set.
Sign Language Recognition using 3D convolutional neural networks

Sign Language Recognition (SLR) targets on interpreting the sign language into text or speech, so as to facilitate the communication between deaf-mute people and ordinary people. This task has broad social impact, but is still very challenging due to the complexity and large variations in hand actions. Existing methods for SLR use hand-crafted features to describe sign language motion and build classification models based on those features. However, it is difficult to design reliable features to adapt to the large variations of hand gestures. To approach this problem, we propose a novel 3D convolutional neural network (CNN) which extracts discriminative spatial-temporal features from raw video stream automatically without any prior knowledge, avoiding designing features. To boost the performance, multi-channels of video streams, including color information, depth clue, and body joint positions, are used as input to the 3D CNN in order to integrate color, depth and trajectory information. We validate the proposed model on a real dataset collected with Microsoft Kinect and demonstrate its effectiveness over the traditional approaches based on hand-crafted features.
Congestion-aware transmission rate control using Medium Grain Scalability of Scalable Video Coding

In packet-oriented networks, packet losses occur mainly due to queue overflows in congested network elements. An increase of the packet rate therefore raises the likelihood of congestion and expected number of lost packets. However, an increase of the packet size has typically a negligible impact on the packet loss rate or congestion in wired packet-switched networks. Scalable Video Coding (SVC), an extension of the Advanced Video Coding (H.264/AVC), provides different types of scalability, one of which is Medium Grain Quality Scalability (MGS). When MGS is in use, layer representations can be pruned unevenly without affecting the decoding of the remaining bitstream. The article proposes a packetization algorithm to improve the quality of the reconstructed video stream while the expected packet loss rate remains unchanged compared to conventional packetization. The algorithm is based on appending MGS enhancement layer data into conventionally generated packet payloads until the Maximum Transmission Unit (MTU) is reached. The simulation results show that the proposed algorithm provides 0.3 to 0.5 dB gain in average luma Peak Signal-to-Noise Ratio when compared with a conventional packetization method while the packet rate remains unchanged.
Rate-Distortion Optimized Reference Picture Management for High Efficiency Video Coding

Motion compensation with multiple reference pictures has been widely used during the development of the emerging High Efficiency Video Coding (HEVC) standard, which greatly helps to improve the coding efficiency. Usually, a heuristic strategy is exploited to use the nearest reconstructed pictures as references. However, such a strategy may not be efficient on all occasions, especially when different content characteristics and coding settings are considered. In this paper, we investigate how to manage reference pictures so as to achieve better rate-distortion performance under the memory constraint of the decoded picture buffer at the decoder. We formulate the reference picture management as an optimization problem and approximate its optimal solution. Moreover, we explore how to adjust quality for each picture according to the reference structure to further improve coding efficiency. For some coding cases, where a complicated encoder optimization is unaffordable, we also develop fast algorithms to get the most benefit from reference picture selection. Among them, one strategy has been adopted by the HEVC software and common test conditions to generate the anchor. Experimental results show that the proposed full search algorithm and fast search algorithms achieve significant bitrate reduction.