Miracle-dz/UniFAR-FTUs huggingface dataset:
OCR-Pose: Occlusion-aware Contrastive Representation for Unsupervised 3D Human Pose Estimation

Occlusion is a significant problem in 3D human pose estimation from the 2D counterpart. On one hand, without explicit annotation, the 3D skeleton is hard to be accurately estimated from the occluded 2D pose. On the other hand, one occluded 2D pose might correspond to multiple 3D skeletons with low confidence parts. To address these issues, we decouple the 3D representation feature into view-invariant part termed occlusion-aware feature and view-dependent part termed rotation feature to facilitate subsequent optimization of the former. Then we propose an occlusion-aware contrastive representation based scheme (OCR-Pose) consisting of Topology Invariant Contrastive Learning module (TiCLR) and View Equivariant Contrastive Learning module (VeCLR). Specifically, TiCLR drives invariance to topology transformation, i.e., bridging the gap between an occluded 2D pose and the unoccluded one. While VeCLR encourages equivariance to view transformation, i.e., capturing the geometric similarity of the 3D skeleton in two views. Both modules optimize occlusion-aware constrastive representation with pose filling and lifting networks via an iterative training strategy in an end-to-end manner. OCR-Pose not only achieves superior performance against state-of-the-art unsupervised methods on unoccluded benchmarks, but also obtains significant improvements when occlusion is involved. Our project is available at https://sites.google. com/view/ocr-pose.
Fine-Tuning NER with spaCy for Transliterated Entities Found in Digital Collections From the Multilingual Persian Gulf

Text recognition technologies increase access to global archives and make possible their computational study using techniques such as Named Entity Recognition (NER). In this paper, we present an approach to extracting a variety of named entities (NE) in unstructured historical datasets from open digital collections dealing with a space of informal British empire: the Persian Gulf region. The sources are largely concerned with people, places and tribes as well as economic and diplomatic transactions in the region. Since models in state-of-the-art NER systems function with limited tag sets and are generally trained on English-language media, they struggle to capture entities of interest to the historian and do not perform well with entities transliterated from other languages. We build custom spaCy-based NER models trained on domain-specific annotated datasets. We also extend the set of named entity labels provided by spaCy and focus on detecting entities of non-Western origin, particularly from Arabic and Farsi. We test and compare performance of the blank, pre-trained and merged spaCy-based models, suggesting further improvements. Our study makes an intervention into thinking beyond Western notions of the entity in digital historical research by creating more inclusive models using non-metropolitan corpora in English.
Decoy Effect of Recommendation Systems on Real E-commerce Websites

Recommendations on e-commerce websites help users discover their interests and assist them in deciding on items to purchase; however, users are prone to bias when comparing and selecting items due to cognitive limitations. The decoy effect, a common user bias phenomenon, has been confirmed in previous studies to induce user selection of items by adding one other item when comparing two items. Although previous studies have confirmed the difference in item selection with and without decoy items in controlled experiments, the mechanism of decoy effect in e-commerce websites has not been elucidated. This study is the first to propose a method for evaluating the decoy effect on real e-commerce websites. We proposed a row-based decoy effect detection method inspired by users' tendency to compare items in the same row when browsing recommended items on e-commerce websites. In addition, a new metric, called intra-row decoy effect rate , is proposed to evaluate the degree of decoy effect. Our month-long study of the recommended order of items on three e-commerce sites reveals that e-commerce sites influence users' item choices regardless of whether they intentionally generate a decoy effect.
Tracing Signs of Urbanity in the Finnish Fiction Film of the 1950s: Toward a Multimodal Analysis of Audiovisual Data

This paper traces signs of urban culture in Finnish fiction films from the 1950s by drawing on a multimodal analysis of audiovisual content. The Finnish National Filmography includes 208 feature films released between 1950–1959. Our approach to the automatic analysis of media content includes aural and visual object recognition and speech recognition. We concentrate on features that epitomize urbanity, including visual objects, such as forms of transportation (cars, horses) and sounds (rural and urban sounds, speech). Based on the scores and frequencies of these recognitions, we observe quantitative changes that took place during the 1950s. The paper demonstrates that aural and visual object recognition, as well as speech recognition, can successfully be applied in film historical analysis. The overall results support the idea that Finnish filmmakers fueled the imagination of urban life in the 1950s, paving the way for modern technologies and gradually pushing aside the signs of rural life.
DrawMon: A Distributed System for Detection of Atypical Sketch Content in Concurrent Pictionary Games

Pictionary, the popular sketch-based guessing game, provides an opportunity to analyze shared goal cooperative game play in restricted communication settings. However, some players occasionally draw atypical sketch content. While such content is occasionally relevant in the game context, it sometimes represents a rule violation and impairs the game experience. To address such situations in a timely and scalable manner, we introduce DrawMon, a novel distributed framework for automatic detection of atypical sketch content in concurrently occurring Pictionary game sessions. We build specialized online interfaces to collect game session data and annotate atypical sketch content, resulting in AtyPict, the first ever atypical sketch content dataset. We use AtyPict to train CanvasNet, a deep neural atypical content detection network. We utilize CanvasNet as a core component of DrawMon. Our analysis of post deployment game session data indicates DrawMon's effectiveness for scalable monitoring and atypical sketch content detection. Beyond Pictionary, our contributions also serve as a design guide for customized atypical content response systems involving shared and interactive whiteboards. Code and datasets are available at https://drawm0n.github. io.
Contribution of cross-correlations to the 21-cm angular power spectrum in the epoch of re-ionization

Measurement of the 21-cm hyperfine transition of neutral hydrogen provides a unique probe of the epoch of re-ionization and the dark ages. Three major mechanisms are believed to dominate the radiation process: emission from neutral hydrogen surrounding the ionized bubbles of first galaxies and/or quasars, emission from neutral hydrogen inside minihaloes and absorption of diffuse neutral hydrogen against the cosmic microwave background. In the present work, by simply combining the existing analytic models for the three mechanisms, we investigate the contribution of cross-correlation between these three components to the total 21-cm angular power spectrum, in the sense that neutral hydrogen associated with different radiation processes traces the large-scale structures of underlying density perturbations. While the overall 21-cm power spectrum remains almost unchanged with the inclusion of the cross-correlations, the cross-correlation may play a key role in the determination of the 21-cm power spectrum during the transition of 21-cm radiation from emission-dominated phase to absorption-dominated phase at redshift z ≈ 20. A significant suppression in the 21-cm angular power spectrum during this transition is anticipated as the result of negative contribution of the cross-correlation between the absorption of diffuse neutral hydrogen and the emission components. Therefore, an accurate prediction of the cosmic 21-cm power spectrum should take the cross-correlation into account especially at the transition phase.
First-principles DFT study for the structural stability of Hydrogen passivated graphene (H-graphene) and atomic adsorption of oxygen on H-graphene with different schemes

First-principles DFT levels of calculations have been carried out in order to study the structural stability and electronic properties of hydrogen passivated graphene (H-graphene) clusters. Two different shaped clusters, rectangular and circular, consisting of 6 to 160 carbon atoms and hydrogen termination at the zigzag boundary edges have been studied. The relative stability of rectangular shaped cluster consisting 96 C-atoms found to be 1.5% greater than that of circular shape cluster consisting same number of C-atoms. In comparing circular and rectangular cluster containing same number of C-atoms, the HOMO-LUMO gap of former have been predicted to be 2.159 eV and that of later just 0.346 eV. Adsorption of oxygen atom on H-graphene with different schemes including single sided, both sided and high concentration adsorption, was also studied systematically through first-principles DFT calculations by taking four different H-graphene clusters.
One-step Low-Rank Representation for Clustering

Existing low-rank representation-based methods adopt a two-step framework, which must employ an extra clustering method to gain labels after representation learning. In this paper, a novel one-step representation-based method, i.e., One-step Low-Rank Representation (OLRR), is proposed to capture multi-subspace structures for clustering. OLRR integrates the low-rank representation model and clustering into a unified framework. Thus it can jointly learn the low-rank subspace structure embedded in the database and gain the clustering results. In particular, by approximating the representation matrix with two same clustering indicator matrices, OLRR can directly show the probability of samples belonging to each cluster. Further, a probability penalty is introduced to ensure that the samples with smaller distances are more inclined to be in the same cluster, thus enhancing the discrimination of the clustering indicator matrix and resulting in a more favorable clustering performance. Moreover, to enhance the robustness against noise, OLRR uses the probability to guide denoising and then performs representation learning and clustering in a recovered clean space. Extensive experiments well demonstrate the robustness and effectiveness of OLRR. Our code is publicly available at: https://github.com/fuzhiqiang1230/OLRR.
AI-VQA: Visual Question Answering based on Agent Interaction with Interpretability

Visual Question Answering (VQA) serves as a proxy for evaluating the scene understanding of an intelligent agent by answering questions about images. Most VQA benchmarks to date are focused on those questions that can be answered through understanding visual content in the scene, such as simple counting, visual attributes, and even a little challenging questions that require extra encyclopedic knowledge. However, humans have a remarkable capacity to reason dynamic interaction on the scene, which is beyond the literal content of an image and has not been investigated so far. In this paper, we propose Agent Interaction Visual Question Answering (AI-VQA), a task investigating deep scene understanding if the agent takes a certain action. For this task, a model not only needs to answer action-related questions but also to locate the objects in which the interaction occurs for guaranteeing it truly comprehends the action. Accordingly, we make a new dataset based on Visual Genome and ATOMIC knowledge graph, including more than 19,000 manually annotated questions, and will make it publicly available. Besides, we also provide an annotation of the reasoning path while developing the answer for each question. Based on the dataset, we further propose a novel method, called ARE, that can comprehend the interaction and explain the reason based on a given event knowledge base. Experimental results show that our proposed method outperforms the baseline by a clear margin.
PixelSeg: Pixel-by-Pixel Stochastic Semantic Segmentation for Ambiguous Medical Images

Semantic segmentation tasks often have multiple output hypotheses for a single input image. Particularly in medical images, these ambiguities arise from unclear object boundaries or differences in physicians' annotation. Learning the distribution of annotations and automatically giving multiple plausible predictions is useful to assist physicians in their decision-making. In this paper, we propose a semantic segmentation framework, PixelSeg, for modelling aleatoric uncertainty in segmentation maps and generating multiple plausible hypotheses. Unlike existing works, PixelSeg accomplishes the semantic segmentation task by sampling the segmentation maps pixel by pixel, which is achieved by the PixelCNN layers used to capture the conditional distribution between pixels. We propose (1) a hierarchical architecture to model high-resolution segmentation maps more flexibly, (2) a fast autoregressive sampling algorithm to improve sampling efficiency by 96.2, and (3) a resampling module to further improve predictions' quality and diversity. In addition, we demonstrate the great advantages of PixelSeg in the novel area of interactive uncertainty segmentation, which is beyond the capabilities of existing models. Extensive experiments and state-of-the-art results on the LIDC-IDRI and BraTS 2017 datasets demonstrate the effectiveness of our proposed model.
DPCNet: Dual Path Multi-Excitation Collaborative Network for Facial Expression Representation Learning in Videos

Current works of facial expression learning in video consume significant computational resources to learn spatial channel feature representations and temporal relationships. To mitigate this issue, we propose a Dual Path multi-excitation Collaborative Network (DPCNet) to learn the critical information for facial expression representation from fewer keyframes in videos. Specifically, the DPCNet learns the important regions and keyframes from a tuple of four view-grouped frames by multi-excitation modules and produces dual-path representations of one video with consistency under two regularization strategies. A spatial-frame excitation module and a channel-temporal aggregation module are introduced consecutively to learn spatial-frame representation and generate complementary channel-temporal aggregation, respectively. Moreover, we design a multi-frame regularization loss to enforce the representation of multiple frames in the dual view to be semantically coherent. To obtain consistent prediction probabilities from the dual path, we further propose a dual path regularization loss, aiming to minimize the divergence between the distributions of two-path embeddings. Extensive experiments and ablation studies show that the DPCNet can significantly improve the performance of video-based FER and achieve state-of-the-art results on the large-scale DFEW dataset.
Recipe-oriented Food Logging for Nutritional Management

We propose a recipe-oriented food logging method that records food by recipe, unlike the ordinary food logging method that records food by name. We also develop an application RecipeLog for this purpose. RecipeLog can create a "skeleton recipe," which is a standardized recipe representation suitable for estimating the nutritional value of a dish. This is represented by a list of ingredients linked to a Nutrition Facts table and a flow graph consisting of cooking actions such as cutting, mixing, baking, simmering, and frying the ingredients. The recipe log allows recipes to be written with fewer operations by editing only the differences from the already registered base recipe. Experiments have confirmed that the recipe log can effectively identify differences in recipes from household to household. The future work is to construct a multimedia recipe dataset consisting of structured recipes and their images using RecipeLog.
Tomograms of spinning black holes

The classical internal structure of spinning black holes is vastly different from that of static black holes. We consider spinning Banados-Teitelboim-Zanelli black holes, and probe their interior from the gauge theory. Utilizing the simplicity of the geometry and reverse engineering from the geodesics, we propose a thermal correlator construction which can be interpreted as arising from two entangled conformal field theories. By analytic continuation of these correlators, we can probe the Cauchy horizon. Correlators that capture the Cauchy horizon in our work have a structure closely related to those that capture the singularity in a nonrotating Banados-Teitelboim-Zanelli. As expected, the regions beyond the Cauchy horizon are not probed in this picture, protecting cosmic censorship.
Hierarchical Walking Transformer for Object Re-Identification

Recently, transformer purely based on attention mechanism has been applied to a wide range of tasks and achieved impressive performance. Though extensive efforts have been made, there are still drawbacks to the transformer architecture which hinder its further applications: (i) the quadratic complexity brought by attention mechanism; (ii) barely incorporated inductive bias. In this paper, we present a new hierarchical walking attention, which provides a scalable, flexible, and interpretable sparsification strategy to reduce the complexity from quadratic to linear, and meanwhile evidently boost the performance. Specifically, we learn a hierarchical structure by splitting an image with different receptive fields. We associate each high-level region with a supernode, and inject supervision with prior knowledge in this node. Supernode then acts as an indicator to decide whether this area should be skipped and thereby massive unnecessary dot-product terms in attention can be avoided. Two sparsification phases are finally introduced, allowing the transformer to achieve strictly linear complexity. Extensive experiments are conducted to demonstrate the superior performance and efficiency against state-of-the-art methods. Significantly, our method sharply reduces the inference time and the total of tokens by 28% and $94%$ respectively, and brings 2.6%@Rank-1 promotion on MSMT17.
The Theory of a Free Jet of a Compressible Gas

In the present report the theory of free turbulence propagation and the boundary layer theory are developed for a plane-parallel free stream of a compressible fluid. In constructing the theory use was made of the turbulence hypothesis by Taylor (transport of vorticity) which gives best agreement with test results for problems involving heat transfer in free jets.
DetFusion: A Detection-driven Infrared and Visible Image Fusion Network

Infrared and visible image fusion aims to utilize the complementary information between the two modalities to synthesize a new image containing richer information. Most existing works have focused on how to better fuse the pixel-level details from both modalities in terms of contrast and texture, yet ignoring the fact that the significance of image fusion is to better serve downstream tasks. For object detection tasks, object-related information in images is often more valuable than focusing on the pixel-level details of images alone. To fill this gap, we propose a detection-driven infrared and visible image fusion network, termed DetFusion, which utilizes object-related information learned in the object detection networks to guide multimodal image fusion. We cascade the image fusion network with the detection networks of both modalities and use the detection loss of the fused images to provide guidance on task-related information for the optimization of the image fusion network. Considering that the object locations provide a priori information for image fusion, we propose an object-aware content loss that motivates the fusion model to better learn the pixel-level information in infrared and visible images. Moreover, we design a shared attention module to motivate the fusion network to learn object-specific information from the object detection networks. Extensive experiments show that our DetFusion outperforms state-of-the-art methods in maintaining pixel intensity distribution and preserving texture details. More notably, the performance comparison with state-of-the-art image fusion methods in task-driven evaluation also demonstrates the superiority of the proposed method. Our code will be available: https://github.com/SunYM2020/DetFusion.
Learned Internet Congestion Control for Short Video Uploading

Short video uploading service has become increasingly important, as at least 30 million videos are uploaded per day. However, we find that existing congestion control (CC) algorithms, either heuristics or learning-based, are not applicable for video uploading -- i.e., lacking in the design of the fundamental mechanism and being short of leveraging network modeling. We present DuGu, a novel learning-based CC algorithm designed by considering the unique proprieties of video uploading via the probing phase and internet networking via the control phase. During the probing phase, DuGu leverages the transmission gap of uploading short videos to actively detect the network metrics to better understand network dynamics. DuGu uses a neural network~(NN) to avoid congestion during the control phase. Here, instead of using handcrafted reward functions, the NN is learned by imitating the expert policy given by the optimal solver, improving both performance and learning efficiency. To build this system, we construct an omniscient-like network emulator, implement an optimal solver and collect a large corpus of real-world network traces to learn expert strategies. Trace-driven and real-world A/B tests reveal that DuGu supports multi-objective and rivals or outperforms existing CC algorithms across all considered scenarios.
OpenHardwareVC: An Open Source Library for 8K UHD Video Coding Hardware Implementation

The hardware-accelerated real-time compression of 8K Ultra-High-Definition (UHD) video is an exemplary application that empowered by the latest video coding standard. However, the coding tools added to the recently released third-generation audio video coding standard (AVS3) greatly increase the coding complexity, which seriously hinders the efficient implementation of hardware encoder. In order to break the known bottleneck, this paper presents the first open source software library for 8K UHD video coding hardware implementation, namely OpenHardwareVC. Specifically, based on the analysis of the original AVS3 software algorithm, we provide the hardware acceleration designs of the four major coding stages, including coding unit (CU) partition, intra prediction, transform and entropy coding, in this library. Simulation results on Xilinx VU440 FPGA show that the real-time compression of 8K UHD videos at 30 frames per second (fps) can be easily supported based on software-described modules packaged in this library. The release of this library is quite favorable for the hardware design and system implementation of UHD video coding, which is also beneficial to the promotion of the new coding standard. The open source library for OpenHardwareVC is available at https://git.openi. org.cn/OpenHardwareVC.