Miracle-dz/UniFAR-FTUs huggingface dataset:
User Centered Adaptive Streaming of Dynamic Point Clouds with Low Complexity Tiling

In recent years, the development of devices for acquisition and rendering of 3D contents have facilitated the diffusion of immersive virtual reality experiences. In particular, the point cloud representation has emerged as a popular format for volumetric photorealistic reconstructions of dynamic real world objects, due to its simplicity and versatility. To optimize the delivery of the large amount of data needed to provide these experiences, adaptive streaming over HTTP is a promising solution. In order to ensure the best quality of experience within the bandwidth constraints, adaptive streaming is combined with tiling to optimize the quality of what is being visualized by the user at a given moment; as such, it has been successfully used in the past for omnidirectional contents. However, its adoption to the point cloud streaming scenario has only been studied to optimize multi-object delivery. In this paper, we present a low-complexity tiling approach to perform adaptive streaming of point cloud content. Tiles are defined by segmenting each point cloud object in several parts, which are then independently encoded. In order to evaluate the approach, we first collect real navigation paths, obtained through a user study in 6 degrees of freedom with 26 participants. The variation in movements and interaction behaviour among users indicate that a user-centered adaptive delivery could lead to sensible gains in terms of perceived quality. Evaluation of the performance of the proposed tiling approach against state of the art solutions for point cloud compression, performed on the collected navigation paths, confirms that considerable gains can be obtained by exploiting user-adaptive streaming, achieving bitrate gains up to 57% with respect to a non-adaptive approach with the same codec. Moreover, we demonstrate that the selection of navigation data has an impact on the relative objective scores.
Fonts Like This but Happier: A New Way to Discover Fonts

Fonts carry strong emotional and social signals, and can affect user engagement in significant ways. Hence, selecting the right font is a very important step in the design of a multimodal artifact with text. Currently, font exploration is frequently carried out via associated social tags. Users are expected to browse through thousands of fonts tagged with certain concepts to find the one that works best for their use case. In this study, we propose a new multimodal font discovery method in which users provide a reference font together with the changes they wish to obtain in order to get closer to their ideal font. This allows for efficient and goal-driven navigation of the font space, and discovery of fonts that would otherwise likely be missed. We achieve this by learning cross-modal vector representations that connect fonts and query words.
SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

The image generation model based on generative adversarial networks has recently received significant attention and can produce diverse, sharp, and realistic images. However, generating high-resolution images has long been a challenge. In this paper, we propose a progressive spatial recursive adversarial expansion model(called SpatialGAN) capable of producing high-quality samples of the natural image. Our approach uses a cascade of convolutional networks to progressively generate images in a part-to-whole fashion. At each level of spatial expansion, a separate image-to-image spatial adversarial expansion network (conditional GAN) is recursively trained based on context image generated by previous GAN or CGAN. Unlike other coarse-to-fine generative methods that constraint on generative process either by multi-scale resolution or by hierarchical feature, the SpatialGAN decomposes image space into multiple subspaces and gradually resolves uncertainties in the local-to-whole generative process. The SpatialGAN greatly stabilizes and speeds up the training, which allows us to produce images of high quality. Based on visual Inception Score and Fréchet Inception Distance, we demonstrate that the quality of images generated by SpatialGAN on several typical datasets is better than that of images generated by GANs without cascading and comparative with the state of art methods with cascading.
Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

One essential problem in skeleton-based action recognition is how to extract discriminative features over all skeleton joints. However, the complexity of the State-Of-The-Art (SOTA) models of this task tends to be exceedingly sophisticated and over-parameterized, where the low efficiency in model training and inference has obstructed the development in the field, especially for large-scale action datasets. In this work, we propose an efficient but strong baseline based on Graph Convolutional Network (GCN), where three main improvements are aggregated, i.e., early fused Multiple Input Branches (MIB), Residual GCN (ResGCN) with bottleneck structure and Part-wise Attention (PartAtt) block. Firstly, an MIB is designed to enrich informative skeleton features and remain compact representations at an early fusion stage. Then, inspired by the success of the ResNet architecture in Convolutional Neural Network (CNN), a ResGCN module is introduced in GCN to alleviate computational costs and reduce learning difficulties in model training while maintain the model accuracy. Finally, a PartAtt block is proposed to discover the most essential body parts over a whole action sequence and obtain more explainable representations for different skeleton action sequences. Extensive experiments on two large-scale datasets, i.e., NTU RGB+D 60 and 120, validate that the proposed baseline slightly outperforms other SOTA models and meanwhile requires much fewer parameters during training and inference procedures, e.g., at most 34 times less than DGNN, which is one of the best SOTA methods.
Cross Corpus Physiological-based Emotion Recognition Using a Learnable Visual Semantic Graph Convolutional Network

Affective media videos have been used as stimulus to investigate an individual's affective-physio responses. In this study, we aim to develop a network learning strategy for robust cross-corpus emotion recognition using physiological features jointly with affective video content. Specifically, we present a novel framework of Visual Semantic Graph Learning Convolutional Network (VGLCN) for individual emotional state recognition using physiology on transfer learning tasks. The stimulus of videos content is integrated into learnable graph structure to weight the importance of physiology on the two emotion dimensions, valence and arousal. Furthermore, we evaluate our proposed framework on two public emotion databases with a rigorous cross validation method, and our model achieves the best unweighted average recall (UAR), which is 67.9%, 56.9% for arousal and 79.8%, 70.4% for valence on the cross datasets recognition experiments respectively. Further analyses reveal that 1) VGLCN is especially effective on transfer valence binary-task, 2) the physiological features (ECG, EDA) are very informative features for emotion recognition and 3) the affective media videos are important constraint to be included in the framework to stabilize the performance power.
Pushing the Limits of Transmission Concurrency for Low Power Wireless Networks

Concurrent transmission (CT) has been widely adopted to optimize the throughput of various data transmissions in wireless networks, such as bulk data dissemination and high-rate data collection. In CT, besides the possible data frame collision at receivers, we observe that acknowledgment frame (ACK) collision at senders can also significantly diminish concurrency opportunities. In this article, to avoid the potential ACK collision in CT, we propose ALIGNER which develops a new transmission pattern to coordinate concurrent senders in a distributed manner. The key idea is to align the silent periods of concurrent transmitters. To achieve this goal, we align the end of data frames concurrently transmitted by several senders. Therefore, the potentially arriving ACKs can avoid a collision with ongoing data transmissions because the concurrent senders are in a listening state to wait for receivers’ ACKs for a short and fixed period. ALIGNER can be applied for both deterministic and opportunistic forwarding protocols. It optionally uses a random back-off and slotted ACK mechanism to avoid a potential collision among simultaneously arrived ACKs in opportunistic forwarding. In addition, ALIGNER adopts a tailor-made metrics to analyze the throughput benefit of concurrent transmission for both deterministic and opportunistic data collection protocols. We have implemented ALIGNER in TinyOS and conducted extensive experiments on a real testbed. Experimental results show that ALIGNER can significantly increase the concurrency opportunities in both deterministic (up to 105%) and opportunistic (up to 89.7%) forwarding compared with the state-of-the-art CT methods.
Visually Precise Query

We present the problem of Visually Precise Query (VPQ) generation which enables a more intuitive match between a user's information need and an e-commerce site's product description. Given an image of a fashion item, what is the most optimum search query that will retrieve the exact same or closely related product(s) with high probability. In this paper we introduce the task of VPQ generation which takes a product image and its title as its input and provides aword level extractive summary of the title, containing a list of salient attributes, which can now be used as a query to search for similar products. We collect a large dataset of fashion images and their titles and merge it with an existing research dataset which was created for a different task. Given the image and title pair, VPQ problem is posed as identifying a non-contiguous collection of spans within the title. We provide a dataset of around 400K image, title and corresponding VPQ entries and release it to the research community. We provide a detailed description of the data collection process as well as discuss the future direction of research for the problem introduced in this work. We provide the standard text as well as visual domain baseline comparisons and also provide multi-modal baseline models to analyze the task introduced in this work. Finally, we propose a hybrid fusion model which promises to be the direction of research in the multi-modal community.
Look, Listen and Infer

Inspired by the ability of human beings on recognizing the relations between visual scenes and sounds, many cross-modal learning methods have been developed for modeling images or videos and associated sounds. In this work, for the first time, a Look, Listen and Infer Network (LLINet) is proposed to learn a zero-shot model that can infer the relations of visual scenes and sounds from novel categories never appeared before. LLINet is mainly desired to qualify for two tasks, i.e., image-audio cross-modal retrieval and sound localization in each image. Towards this end, it is designed as a two-branch encoding network that builds a common space for images and audios. Besides, a cross-modal attention mechanism is proposed in LLINet to localize sound objects. To evaluate LLINet, a new data set, named INSTRUMENT-32CLASS, is collected in this work. Besides zero-shot cross-modal retrieval and sound localization, a zero-shot image recognition task based on sounds is also conducted on this database. All experimental results on these tasks demonstrate the effectiveness of LLINet, indicating that zero-shot learning for visual scenes and sounds is feasible. The project page for LLINet is available at https://llinet.github. io/.
SST-EmotionNet: Spatial-Spectral-Temporal based Attention 3D Dense Network for EEG Emotion Recognition

Multimedia stimulation of brain activities has not only become an emerging field for intensive research, but also achieves important progress in the electroencephalogram (EEG) emotion classification based on brain activities. However, how to make full use of different EEG features and the discriminative local patterns among the features for different emotions is challenging. Existing models ignore the complementarity among the spatial-spectral-temporal features and discriminative local patterns in all features, which limits the classification ability of the models to a certain extent. In this paper, we propose a novel spatial-spectral-temporal based attention 3D dense network, named SST-EmotionNet, for EEG emotion recognition. The main advantage of the SST-EmotionNet is the simultaneous integration of spatial-spectral-temporal features in a unified network framework. Meanwhile, a 3D attention mechanism is designed to adaptively explore discriminative local patterns. Extensive experiments on two real-world datasets demonstrate that the SST-EmotionNet outperforms the state-of-the-art baselines.
CF-SIS: Semantic-Instance Segmentation of 3D Point Clouds by Context Fusion with Self-Attention

3D Semantic-Instance Segmentation (SIS) is a newly emerging research direction that aims to understand visual information of 3D scene on both semantic and instance level. The main difficulty lies in how to coordinate the paradox between mutual aid and sub-optimal problem. Previous methods usually address the mutual aid between instances and semantics by direct feature fusion or hand-crafted constraints to share the common knowledge of the two tasks. However, they neglect the abundant common knowledge of feature context in the feature space. Moreover, the direct feature fusion can raise the sub-optimal problem, since the false prediction of instance object can interfere the prediction of the semantic segmentation and vice versa. To address the above two issues, we propose a novel network of feature context fusion for SIS task, named CF-SIS. The idea is to associatively learn semantic and instance segmentation of 3D point clouds by context fusion with attention in the feature space. Our main contributions are two context fusion modules. First, we propose a novel inter-task context fusion module to take full advantage of mutual aid and relive the sub-optimal problem. It extracts the context in feature space from one task with attention, and selectively fuses the context into the other task using a gate fusion mechanism. Then, in order to enhance the mutual aid effect, the intra-task context fusion module is designed to further integrate the fused context, by selectively merging the similar feature through the self-attention mechanism. We conduct experiments on the S3DIS and ShapeNet datasets and show that CF-SIS outperforms the state-of-the-art methods on semantic and instance segmentation task.
A Novel Graph-TCN with a Graph Structured Representation for Micro-expression Recognition

Facial micro-expressions (MEs) recognition has attracted much attention recently. However, because MEs are spontaneous, subtle and transient, recognizing MEs is a challenge task. In this paper, first, we use transfer learning to apply learning-based video motion magnification to magnify MEs and extract the shape information, aiming to solve the problem of the low muscle movement intensity of MEs. Then, we design a novel graph-temporal convolutional network (Graph-TCN) to extract the features of the local muscle movements of MEs. First, we define a graph structure based on the facial landmarks. Second, the Graph-TCN deals with the graph structure in dual channels with a TCN block. One channel is for node feature extraction, and the other one is for edge feature extraction. Last, the edges and nodes are fused for classification. The Graph-TCN can automatically train the graph representation to distinguish MEs while not using a hand-crafted graph representation. To the best of our knowledge, we are the first to use the learning-based video motion magnification method to extract the features of shape representations from the intermediate layer while magnifying MEs. Furthermore, we are also the first to use deep learning to automatically train the graph representation for MEs.
EyeShopper: Estimating Shoppers' Gaze using CCTV Cameras

Recent advances in machine and deep learning allow for enhanced retail analytics by applying object detection techniques. However, existing approaches either require laborious installation processes to function or lack precision when the customers turn their back in the installed cameras. In this paper, we present EyeShopper, an innovative system that tracks the gaze of shoppers when facing away from the camera and provides insights about their behavior in physical stores. EyeShopper is readily deployable in existing surveillance systems and robust against low-resolution video inputs. At the same time, its accuracy is comparable to state-of-the-art gaze estimation frameworks that require high-resolution and continuous video inputs to function. Furthermore, EyeShopper is more robust than state-of-the-art gaze tracking techniques for back head images. Extensive evaluation with different real video datasets and a synthetic dataset we produced shows that EyeShopper estimates with high accuracy the gaze of customers.
The inhibitory effect of the curcumol on women cancer cells and synthesis of RNA

Objective The study was designed to explore the inhibitory effect of curcumol from Curcuma on the proliferation of tumor cells. Methods The effect of curcumol on the proliferation of 15 cell lines (13 women's tumor cells and 2 normal mammary gland cells)was in vitro examined by using MTT assay, and synthesis of RNA was observed. Results The proliferation of tumour cells was remarkably inhibited and synthesis of RNA was evident because of induction by curcumol. Conclusion Curcumol inhibits the proliferation of women's tumor cells (MCF7, MM231, HeLa and OV-UL-2) in vitro, and synthesis of RNA (MCF7,MM231,HeLa) was evident.
Sketch‐based modeling with a differentiable renderer

Sketch‐based modeling aims to recover three‐dimensional (3D) shape from two‐dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end‐to‐end learning framework to predict 3D shape from line drawings. Our approach is based on a two‐steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state‐of‐art, which demonstrates that our framework is able to cope with the challenges in single sketch‐based 3D shape modeling.
Owning, Using, and Renting: Some Simple Economics of the "Sharing Economy"

New Internet-based markets enable consumer/owners to rent out their durable goods when not using them. Such markets are modeled to determine ownership, rental rates, quantities, and surplus generated. Both the short run, before consumers can revise their ownership decisions, and the long run, in which they can, are examined to assess how these markets change ownership and consumption. The analysis examines bringing-to-market costs, such as labor costs and transaction costs, and considers the operating platform’s pricing problem. A survey of consumers broadly supports the modeling assumptions employed. For example, ownership is determined by individuals’ forward-looking assessments of planned usage. Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber. org.
Equivalent efficacies of reverse hybrid and concomitant therapies in first‐line treatment of Helicobacter pylori infection

Concomitant therapy is a recommended first‐line treatment for Helicobacter pylori infection in most national or international consensuses. Reverse hybrid therapy is a modified 14‐day concomitant therapy without clarithromycin and metronidazole in the final 7 days. This study aims to test whether 14‐day reverse hybrid therapy is non‐inferior to 14‐day concomitant therapy in the first‐line treatment of H. pylori infection.
Effect of etiolation on polyphenol oxidase activity in shoots of grape and its subsequent in vitro survival

Etiolation treatment was given to seven different grape genotypes after winter pruning to observe its effect on total phenols, PPO activity and on in vitro survival of shoots tips. Total phenol content and PPO activity was reduced dramatically in etiolated shoots over non-etiolated which was found to be genotypic dependent. Etiolation treatment ultimately resulted in marked increase in explant survival. Synergistic effect of genotype and etiolation treatment was found in lowering the endogenous level of polyphenol content, PPO activity and increasing explant survival in in vitro.
Graph colour-based resource allocation for relay-assisted D2D underlay communications

Relay-assisted device-to-device (D2D) communications have been proposed as a supplement for direct D2D communications to enhance traffic offloading capacity. In this study, the authors propose a joint mode selection, relay selection and resource allocation for relay-assisted D2D communications. They aim at maximising the overall system throughput while guaranteeing the power limitation and signal-to-noise-and-interference ratios of all cellular and active D2D links. Since this optimisation is NP-hard, they then propose a graph colour-based resource allocation algorithm to effectively solve it. Simulation results show that the proposed algorithm can produce close-to-optimal performance with acceptable computational complexity.
Expressions of EF-Tu_( mt) and EF-Ts_(mt) in brain of rat with epilepsg

Objective: To investigate the relationship between mitochondrial translation elongation factors Tu and Ts (EF-Tu mt, EF-Ts mt) and status epilepticus in rats. Methods: Based on the rat model of status epilepticus, the expressions of EF-Tu mt and EF-Ts mt were observed by using immunohistochemical method. Results: There were higher expressions of EF-Tu mt and EF-Ts mt in all the seizure groups compared with that in the normal group. It happened in the brain cortex, hippocampus and striatum. Furthermore, in brain cortex and hippocampus the alterative trend of EF-Ts mt with time was the same as EF-Tu mt. Conclusion: The expressions of EF-Tu mt and EF-Ts mt in brain cortex, hippocampus and striatum become higher with the progression of status epilepticus.