Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Joint Multimodal Representation with Adversarial Attention Networks

Recently, learning a joint representation for the multimodal data (e.g., containing both visual content and text description) has attracted extensive research interests. Usually, the features of different modalities are correlational and compositive, and thus a joint representation capturing the correlation is more effective than a subset of the features. Most of existing multimodal representation learning methods suffer from lack of additional constraints to enhance the robustness of the learned representations. In this paper, a novel Adversarial Attention Networks (AAN) is proposed to incorporate both the attention mechanism and the adversarial networks for effective and robust multimodal representation learning. Specifically, a visual-semantic attention model with siamese learning strategy is proposed to encode the fine-grained correlation between visual and textual modalities. Meanwhile, the adversarial learning model is employed to regularize the generated representation by matching the posterior distribution of the representation to the given priors. Then, the two modules are incorporated into a integrated learning framework to learn the joint multimodal representation. Experimental results in two tasks, i.e., multi-label classification and tag recommendation, show that the proposed model outperforms state-of-the-art representation learning methods.
Hierarchical Memory Modelling for Video Captioning

Translating videos into natural language sentences has drawn much attention recently. The framework of combining visual attention with Long Short-Term Memory (LSTM) based text decoder has achieved much progress. However, the vision-language translation still remains unsolved due to the semantic gap and misalignment between video content and described semantic concept. In this paper, we propose a Hierarchical Memory Model (HMM) - a novel deep video captioning architecture which unifies a textual memory, a visual memory and an attribute memory in a hierarchical way. These memories can guide attention for efficient video representation extraction and semantic attribute selection in addition to modelling the long-term dependency for video sequence and sentences, respectively. Compared with traditional vision-based text decoder, the proposed attribute-based text decoder can largely reduce the semantic discrepancy between video and sentence. To prove the effectiveness of the proposed model, we perform extensive experiments on two public benchmark datasets: MSVD and MSR-VTT. Experiments show that our model not only can discover appropriate video representation and semantic attributes but also can achieve comparable or superior performances than state-of-the-art methods on these datasets.
LA-Net: Layout-Aware Dense Network for Monocular Depth Estimation

Depth estimation from monocular images is an ill-posed and inherently ambiguous problem. Recently, deep learning technique has been applied for monocular depth estimation seeking data-driven solutions. However, most existing methods focus on pursuing the minimization of average depth regression error at pixel level and neglect to encode the global layout of scene, resulting in layout-inconsistent depth map. This paper proposes a novel Layout-Aware Convolutional Neural Network (LA-Net) for accurate monocular depth estimation by simultaneously perceiving scene layout and local depth details. Specifically, a Spatial Layout Network (SL-Net) is proposed to learn a layout map representing the depth ordering between local patches. A Layout-Aware Depth Estimation Network (LDE-Net) is proposed to estimate pixel-level depth details using multi-scale layout maps as structural guidance, leading to layout-consistent depth map. A dense network module is used as the base network to learn effective visual details resorting to dense feed-forward connections. Moreover, we formulate an order-sensitive softmax loss to well constrain the ill-posed depth inferring problem. Extensive experiments on both indoor scene (NYUD-v2) and outdoor scene (Make3D) datasets have demonstrated that the proposed LA-Net outperforms the state-of-the-art methods and leads to faithful 3D projections.
Explore Multi-Step Reasoning in Video Question Answering

This invited talk is a repeated but more detailed talk about the paper which is accepted by ACM-MM 2018: Video question answering (VideoQA) always involves visual reasoning. When answering questions composing of multiple logic correlations, models need to perform multi-step reasoning. In this paper, we formulate multi-step reasoning in VideoQA as a new task to answer compositional and logical structured questions based on video content. Existing VideoQA datasets are inadequate as benchmarks for the multi-step reasoning due to limitations as lacking logical structure and having language biases. Thus we design a system to automatically generate a large-scale dataset, namely SVQA (Synthetic Video Question Answering). Compared with other VideoQA datasets, SVQA contains exclusively long and structured questions with various spatial and temporal relations between objects. More importantly, questions in SVQA can be decomposed into human readable logical tree or chain layouts, each node of which represents a sub-task requiring a reasoning operation such as comparison or arithmetic. Towards automatic question answering in SVQA, we develop a new VideoQA model. Particularly, we construct a new attention module, which contains spatial attention mechanism to address crucial and multiple logical sub-tasks embedded in questions, as well as a refined GRU called ta-GRU (temporal-attention GRU) to capture the long-term temporal dependency and gather complete visual cues. Experimental results show the capability of multi-step reasoning of SVQA and the effectiveness of our model when compared with other existing models.
βTrCP is Required for HIV-1 Vpu Modulation of CD4, GaLV Env, and BST-2/Tetherin

The Human immunodeficiency virus-1 (HIV-1) accessory protein Vpu modulates numerous proteins, including the host proteins CD4 and BST-2/tetherin. Vpu interacts with the Skp, Cullin, F-Box (SCF) ubiquitin ligase through interactions with the F-Box protein βTrCP (1 and/or 2). This interaction is dependent on phosphorylation of S52,56 in Vpu. Mutation of S52,56, or inhibition of the SCF, abolishes most Vpu activity against CD4 and partly reduces activity against BST-2/tetherin. Recently, Vpu has also been reported to interact with the clathrin adapter proteins AP-1 and AP-2, and these interactions were also found to be required for BST-2/tetherin antagonism in an S52,56 -dependent manner. In assays where HIV-1 is pseudotyped with gibbon ape leukemia virus (GaLV Env), Vpu has also been found to prevent GaLV Env from being incorporated into viral particles, but the mechanism for this antagonism is not fully understood. To clarify the role of the βTrCPs in Vpu function we used CRISPR/Cas9 to generate a clonal cell line lacking both βTrCP-1 and -2. Vpu activity against CD4 and GaLV Env was abolished in this cell line, and activity against BST-2/tetherin reduced significantly. Mutation of the S52,56 residues no longer affected Vpu activity against BST-2/tetherin in this cell line. These data suggest that the primary role of the S52,56 residues in antagonism of CD4, GaLV Env, and BST-2/tetherin is to recruit the SCF/βTrCP ubiquitin ligase.
Kidney Biopsy-Related Complications in Hospitalized Patients with Acute Kidney Disease.

BACKGROUND AND OBJECTIVES Patients are informed of the risk of kidney biopsy-related complications using data from nonhospitalized patients, which may underestimate the risk for hospitalized patients. We evaluated the rate and risk factors of kidney biopsy-related complications in hospitalized patients with acute kidney disease (AKD) to better estimate the risk in this population. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS We used data from the Yale biopsy cohort to evaluate rates of kidney biopsy-related complications including adjudicated procedure-related bleeding requiring blood transfusions or angiographic interventions, medium- or large-sized hematomas, reimaging after biopsy including abdominal ultrasonography or computed tomography, and death in hospitalized patients with AKD (including AKI). We evaluated univariable and multivariable association of risk factors with transfusions. We compared rates of complications between hospitalized and nonhospitalized patients. RESULTS Between 2015 and 2017, 159 hospitalized patients underwent a kidney biopsy for AKD evaluation, of which 80 (51%) had stage 1 AKI, 42 (27%) had stage 2 (or higher) AKI, and 27 (17%) had AKD (without AKI). Of these, 12 (8%; 95% confidence interval [95% CI], 5% to 15%) required a transfusion, three (2%; 95% CI, 1% to 5%) required an intervention, 11 (7%; 95% CI, 4% to 12%) had hematoma, and 31 (20%; 95% CI, 14% to 26%) required reimaging after biopsy. Of the four (3%; 95% CI, 1% to 6%) deaths during hospitalization, none were related to the biopsy. Female sex, lower platelet count, and higher BUN were associated with postbiopsy transfusions on univariable and multivariable analyses. Trainee as proceduralist and larger needle gauge were associated with transfusions in univariable, but not multivariable, analysis. Nonhospitalized patients had lower rates of transfusion than hospitalized patients, although the latter also had lower prebiopsy hemoglobin and greater surveillance after biopsy. CONCLUSIONS Hospitalized patients experience higher risk of postbiopsy complications than previously reported and several factors, such as lower platelet count, female sex, and higher BUN, are associated with this risk.
ORSNet: A Hybrid Neural Network for Official Sports Referee Signal Recognition

In this work, we propose a novel sports referee training system based on wearable sensors and a real-time Official Referee Signal (ORS) segmentation/recognition method which can recognize 65 kinds basketball ORSs with the accuracy of 95.3%. A hybrid neural network named ORSNet is designed for recognizing gestures based on IMU signals. The proposed ORSNet involves convolution layers and recurrent layers to learn more representative features and correlations in temporal domain, respectively. A novel loss function and a weight sharing strategy are proposed to learn a more robust ORS recognition model. Moreover, we investigate the influence of applying a semi-supervised network in the proposed ORSNet.
Validation of the prediction rules identifying drug-resistant pathogens in community-onset pneumonia

Background Appropriate initial antibiotic treatment and avoiding administration of unnecessary broad-spectrum antibiotics are important for the treatment of pneumonia. To achieve this, assessment of risk for drug-resistant pathogens (DRPs) at diagnosis is essential. Purpose The aim of this study was to validate a predictive rule for DRPs that we previously proposed (the community-acquired pneumonia drug-resistant pathogen [CAP-DRP] rule), comparing several other predictive methods. Patients and methods A prospective observational study was conducted in hospitalized patients with community-onset pneumonia at four institutions in Japan. Pathogens identified as not susceptible to ceftriaxone, ampicillin–sulbactam, macrolides, and respiratory fluoroquinolones were defined as CAP-DRPs. Results CAP-DRPs were identified in 73 (10.1%) of 721 patients analyzed. The CAP-DRP rule differentiated low vs high risk of CAP-DRP at the threshold of ≥3 points or 2 points plus any of methicillin-resistant Staphylococcus aureus specific factors with a sensitivity of 0.45, specificity of 0.87, positive predictive value of 0.47, negative predictive value of 0.87, and accuracy of 0.79. Its discrimination performance, area under the receiver operating characteristic curve, was 0.73 (95% confidence interval 0.66–0.79). Specificity of the CAP-DRP rule against CAP-DRPs was the highest among the six predictive rules tested. Conclusion The performance of the predictive rules and criteria for CAP-DRPs was limited. However, the CAP-DRP rule yielded high specificity and could specify patients who should be treated with non-broad-spectrum antibiotics, eg, a non-pseudomonal β-lactam plus a macrolide, more precisely.
StuffIE: Semantic Tagging of Unlabeled Facets Using Fine-Grained Information Extraction

Recent knowledge extraction methods are moving towards ternary and higher-arity relations to capture more information about binary facts. An example is to include the time, the location, and the duration of a specific fact. These relations can be even more complex to extract in advanced domains such as news, where events typically come with different facets including reasons, consequences, purposes, involved parties, and related events. The main challenge consists in first finding the set of facets related to each fact, and second tagging those facets to the relevant category. In this paper, we tackle the above problems by proposing StuffIE, a fine-grained information extraction approach which is facet-centric. We exploit the Stanford dependency parsing enhanced by lexical databases such as WordNet to extract nested triple relations. Then, we exploit the syntactical dependencies to semantically tag facets using distant learning based on Oxford dictionary. We have tested the accuracy of the extracted facets and their semantic tags using DUC'04 dataset. The results show the high accuracy and coverage of our approach with respect to ClausIE, OLLIE, SEMAFOR SRL and Illinois SRL.
Building an emulation environment for cyber security analyses of complex networked systems

Computer networks are undergoing a phenomenal growth, driven by the rapidly increasing number of nodes constituting the networks. At the same time, the number of security threats on Internet and intranet networks is constantly growing, and the testing and experimentation of cyber defense solutions requires the availability of separate, test environments that best emulate the complexity of a real system. Such environments support the deployment and monitoring of complex mission-driven network scenarios, thus enabling the study of cyber defense strategies under real and controllable traffic and attack scenarios. In this paper, we propose a methodology that makes use of a combination of techniques of network and security assessment, and the use of cloud technologies to build an emulation environment with adjustable degree of affinity with respect to actual reference networks or planned systems. As a byproduct, starting from a specific study case, we collected a dataset consisting of complete network traces comprising benign and malicious traffic, which is feature-rich and publicly available.
Clinical Reasoning: West syndrome, pontocerebellar hypoplasia, and hypomyelination in a 6-month-old boy

A 6-month-old boy born to nonconsanguineous parents presented with intractable seizures from the fourth month of age. Oligohydramnios was detected in the last trimester of pregnancy. He was born at term by normal delivery (birthweight 2.9 kg). At birth he had weak cry, microcephaly (head circumference 31 cm, <third centile), micrognathia, prominent nasal bridge, and radial deviation of both hands. Cranial and abdominal ultrasonogram and echocardiogram were normal.
An Effective Approach for Modelling Time Features for Classifying Bursty Topics on Twitter

Several previous approaches attempted to predict bursty topics on Twitter. Such approaches have usually reported that the time information (e.g. the topic popularity over time) of hashtag topics contribute the most to the prediction of bursty topics. In this paper, we propose a novel approach to use time features to predict bursty topics on Twitter. We model the popularity of topics as density curves described by the density function of a beta distribution with different parameters. We then propose various approaches to predict/classify the bursty topics by estimating the parameters of topics, using estimators such as Gradient Decent or Likelihood Maximization. In our experiments, we show that the estimated parameters of topics have a positive effect on classifying bursty topics. In particular, our estimators when combined together improve the bursty topic classification by 6.9 in terms of micro F1 compared to a baseline classifier using hashtag content features.
Mining & Summarizing E-petitions for Enhanced Understanding of Public Opinion

Today electronic communications have become the prime medium for people to express their opinions and influence the policy preferences. One such popular channel reflecting the voice of the masses is electronic petitions. To understand people's perspective on various issues it is important to know what petitions say. However, due to the sheer volume of the petitions it is difficult to process each petition manually. As each petition talks about a different issue, prioritizing on over other is difficult. To alleviate these challenges, we present an end to end system for generating comprehensive and concise summaries from e-petitions. A petition contains multiple aspects, the core problem, evidence(s) in support of the problem and potential solutions. Therefore, it is imperative that an useful summary should contain information about all these three aspects explicitly. To achieve this, our system generates three aspect based summaries for each petition for better understanding. We also introduce a new annotated petition dataset, developed through crowd-sourcing, that served as gold standard. Our model is tested through quantitative and qualitative evaluations.
Compiling Questions into Balanced Quizzes about Documents

In the educational framework, knowledge assessment is a critical component, and quizzes (sets of questions with concise answers) are a popular tool for this purpose. This paper focuses on the generation of balanced quizzes, i.e., quizzes that relate to a given set of documents, and to the central concepts described by the documents, in an evenly distributed manner. Our approach leverages a graph representing the relationships between questions, documents, and concepts, and phrases quiz construction as a node selection problem in this graph. We provide algorithms for constructing the graph and for selecting a good set of quiz questions. In our concrete implementation, we build quizzes for a collection of Wikipedia articles and evaluate them both with simulated students and with real human quiz takers, finding that our balanced quizzes are better suited at determining which articles the user has not read (corresponding to their knowledge gaps) than reasonable baselines.
Deep Learning for Continuous Multiple Time Series Annotations

Learning from multiple annotations is an increasingly important research topic. Compared with conventional classification or regression problems, it faces more challenges because time-continuous annotations would result in noisy and temporal lags problems for continuous emotion recognition. In this paper, we address the problem by deep learning for continuous multiple time series annotations. We attach a novel crowd layer to the output layer of basic continuous emotion recognition system, which learns directly from the noisy labels of multiple annotators with end-to-end manner. The inputs of the system are multimodal features and the targets are multiple annotations, with the intention of learning an annotator-specific mapping. Our proposed method considers the ground truth as latent variables and multiple annotations are variant of ground truth by linear mapping. The experimental results show that our system can achieve superior performance and capture the reliabilities and biases of different annotators.
Bipolar Disorder Recognition with Histogram Features of Arousal and Body Gestures

This paper targets the Bipolar Disorder Challenge (BDC) task of Audio Visual Emotion Challenge (AVEC) 2018. Firstly, two novel features are proposed: 1) a histogram based arousal feature, in which the continuous arousal values are estimated from the audio cues by a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) model; 2) a Histogram of Displacement (HDR) based upper body posture feature, which characterizes the displacement and velocity of the key body points in the video segment. In addition, we propose a multi-stream bipolar disorder classification framework with Deep Neural Networks (DNNs) and a Random Forest, and adopt the ensemble learning strategy to alleviate the possible over-fitting problem due to the limited training data. Experimental results show that the proposed arousal feature and upper body posture feature are discriminative for different bipolar episodes, and our proposed framework achieves promising classification results on the development set, with the unweighted average recall (UAR) of 0.714, which is higher than the baseline result 0.635. On test set evaluation, our system obtains the same UAR (0.574) as the challenge baseline.
Trusted Guidance Pyramid Network for Human Parsing

Human parsing, which segments a human-centric image into pixel-wise categorization, has a wide range of applications. However, none of the existing methods can productively solve the issue of label parsing fragmentation due to confused and complicated annotations. In this paper, we propose a novel Trusted Guidance Pyramid Network (TGPNet) to address this limitation. Based on a pyramid architecture, we design a Pyramid Residual Pooling (PRP) module setting at the end of a bottom-up approach to capture both global and local level context. In the top-down approach, we propose a Trusted Guidance Multi-scale Supervision (TGMS) that efficiently integrates and supervises multi-scale contextual information. Furthermore, we present a simple yet powerful Trusted Guidance Framework (TGF) which imposes global-level semantics into parsing results directly without extra ground truth labels in model training. Extensive experiments on two public human parsing benchmarks well demonstrate that our TGPNet has a strong ability in solving label parsing fragmentation problem and has an obtained improvement than other methods.
Effects of pomegranate supplementation on exercise performance and post-exercise recovery in healthy adults: a systematic review

Abstract The functional significance of pomegranate (POM) supplementation on physiological responses during and following exercise is currently unclear. This systematic review aimed (i) to evaluate the existing literature assessing the effects of POM supplementation on exercise performance and recovery; exercise-induced muscle damage, oxidative stress, inflammation; and cardiovascular function in healthy adults and (ii) to outline the experimental conditions in which POM supplementation is more or less likely to benefit exercise performance and/or recovery. Multiple electronic databases were used to search for studies examining the effects of POM intake on physiological responses during and/or following exercise in healthy adult. Articles were included in the review if they investigated the effects of an acute or chronic POM supplementation on exercise performance, recovery and/or physiological responses during or following exercise. The existing evidence suggests that POM supplementation has the potential to confer antioxidant and anti-inflammatory effects during and following exercise, to improve cardiovascular responses during exercise, and to enhance endurance and strength performance and post-exercise recovery. However, the beneficial effects of POM supplementation appeared to be less likely when (i) unilateral eccentric exercise was employed, (ii) the POM administered was not rich in polyphenols (<1·69 g/l) and (iii) insufficient time was provided between POM-ingestion and the assessment of physiological responses/performance (≤1 h). The review indicates that POM has the potential to enhance exercise performance and to expedite recovery from intensive exercise. The findings and recommendations from this review may help to optimise POM-supplementation practice in athletes and coaches to potentially improve exercise-performance and post-exercise recovery.