OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive Summaries: A Personalized Concept-based Summarization Approach by Learning from Users' Feedback
Venue: arXiv cs.AI 2021
Year: 2021

Exploring the tremendous amount of data efficiently to make a decision, similar to answering a complicated question, is challenging with many real-world application scenarios. In this context, automatic summarization has substantial importance as it will provide the foundation for big data analytic. Traditional summarization approaches optimize the system to produce a short static summary that fits all users that do not consider the subjectivity aspect of summarization, i.e., what is deemed valuable for different users, making these approaches impractical in real-world use cases. This paper proposes an interactive concept-based summarization model, called Adaptive Summaries, that helps users make their desired summary instead of producing a single inflexible summary. The system learns from users' provided information gradually while interacting with the system by giving feedback in an iterative loop. Users can choose either reject or accept action for selecting a concept being included in the summary with the importance of that concept from users' perspectives and confidence level of their feedback. The proposed approach can guarantee interactive speed to keep the user engaged in the process. Furthermore, it eliminates the need for reference summaries, which is a challenging issue for summarization tasks. Evaluations show that Adaptive Summaries helps users make high-quality summaries based on their preferences by maximizing the user-desired content in the generated summaries.
Learning Robust Representation for Clustering through Locality Preserving Variational Discriminative Network
Venue: arXiv cs.LG 2021
Year: 2021

Clustering is one of the fundamental problems in unsupervised learning. Recent deep learning based methods focus on learning clustering oriented representations. Among those methods, Variational Deep Embedding achieves great success in various clustering tasks by specifying a Gaussian Mixture prior to the latent space. However, VaDE suffers from two problems: 1) it is fragile to the input noise; 2) it ignores the locality information between the neighboring data points. In this paper, we propose a joint learning framework that improves VaDE with a robust embedding discriminator and a local structure constraint, which are both helpful to improve the robustness of our model. Experiment results on various vision and textual datasets demonstrate that our method outperforms the state-of-the-art baseline models in all metrics. Further detailed analysis shows that our proposed model is very robust to the adversarial inputs, which is a desirable property for practical applications.
POPO: Pessimistic Offline Policy Optimization
Venue: arXiv cs.LG 2021
Year: 2021

Offline reinforcement learning (RL), also known as batch RL, aims to optimize policy from a large pre-recorded dataset without interaction with the environment. This setting offers the promise of utilizing diverse, pre-collected datasets to obtain policies without costly, risky, active exploration. However, commonly used off-policy algorithms based on Q-learning or actor-critic perform poorly when learning from a static dataset. In this work, we study why off-policy RL methods fail to learn in offline setting from the value function view, and we propose a novel offline RL algorithm that we call Pessimistic Offline Policy Optimization (POPO), which learns a pessimistic value function to get a strong policy. We find that POPO performs surprisingly well and scales to tasks with high-dimensional state and action space, comparing or outperforming several state-of-the-art offline RL algorithms on benchmark tasks.
A coupled finite volume and material point method for two-phase simulation of liquid-sediment and gas-sediment flows
Venue: arXiv cs.CE 2021
Year: 2021

Mixtures of fluids and granular sediments play an important role in many industrial, geotechnical, and aerospace engineering problems, from waste management and transportation (liquid--sediment mixtures) to dust kick-up below helicopter rotors (gas--sediment mixtures). These mixed flows often involve bulk motion of hundreds of billions of individual sediment particles and can contain both highly turbulent regions and static, non-flowing regions. This breadth of phenomena necessitates the use of continuum simulation methods, such as the material point method (MPM), which can accurately capture these large deformations while also tracking the Lagrangian features of the flow (e.g.\ the granular surface, elastic stress, etc.). Recent works using two-phase MPM frameworks to simulate these mixtures have shown substantial promise; however, these approaches are hindered by the numerical limitations of MPM when simulating pure fluids. In addition to the well-known particle ringing instability and difficulty defining inflow/outflow boundary conditions, MPM has a tendency to accumulate quadrature errors as materials deform, increasing the rate of overall error growth as simulations progress. In this work, we present an improved, two-phase continuum simulation framework that uses the finite volume method (FVM) to solve the fluid phase equations of motion and MPM to solve the solid phase equations of motion, substantially reducing the effect of these errors and providing better accuracy and stability for long-duration simulations of these mixtures.
LCCNet: LiDAR and Camera Self-Calibration using Cost Volume Network
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a novel online self-calibration approach for Light Detection and Ranging (LiDAR) and camera sensors. Compared to the previous CNN-based methods that concatenate the feature maps of the RGB image and decalibrated depth image, we exploit the cost volume inspired by the PWC-Net for feature matching. Besides the smooth L1-Loss of the predicted extrinsic calibration parameters, an additional point cloud loss is applied. Instead of regress the extrinsic parameters between LiDAR and camera directly, we predict the decalibrated deviation from initial calibration to the ground truth. During inference, the calibration error decreases further with the usage of iterative refinement and the temporal filtering approach. The evaluation results on the KITTI dataset illustrate that our approach outperforms CNN-based state-of-the-art methods in terms of a mean absolute calibration error of 0.297cm in translation and 0.017° in rotation with miscalibration magnitudes of up to 1.5m and 20°.
Explaining NLP Models via Minimal Contrastive Editing (MiCE)
Venue: arXiv cs.CL 2021
Year: 2021

Humans have been shown to give contrastive explanations, which explain why an observed event happened rather than some other counterfactual event (the contrast case). Despite the influential role that contrastivity plays in how humans explain, this property is largely missing from current methods for explaining NLP models. We present Minimal Contrastive Editing (MiCE), a method for producing contrastive explanations of model predictions in the form of edits to inputs that change model outputs to the contrast case. Our experiments across three tasks--binary sentiment classification, topic classification, and multiple-choice question answering--show that MiCE is able to produce edits that are not only contrastive, but also minimal and fluent, consistent with human contrastive edits. We demonstrate how MiCE edits can be used for two use cases in NLP system development--debugging incorrect model outputs and uncovering dataset artifacts--and thereby illustrate that producing contrastive explanations is a promising research direction for model interpretability.
Pivot Through English: Reliably Answering Multilingual Questions without Document Retrieval
Venue: arXiv cs.CL 2021
Year: 2021

Existing methods for open-retrieval question answering in lower resource languages (LRLs) lag significantly behind English. They not only suffer from the shortcomings of non-English document retrieval, but are reliant on language-specific supervision for either the task or translation. We formulate a task setup more realistic to available resources, that circumvents document retrieval to reliably transfer knowledge from English to lower resource languages. Assuming a strong English question answering model or database, we compare and analyze methods that pivot through English: to map foreign queries to English and then English answers back to target language answers. Within this task setup we propose Reranked Multilingual Maximal Inner Product Search (RM-MIPS), akin to semantic similarity retrieval over the English training set with reranking, which outperforms the strongest baselines by 2.7% on XQuAD and 6.2% on MKQA. Analysis demonstrates the particular efficacy of this strategy over state-of-the-art alternatives in challenging settings: low-resource languages, with extensive distractor data and query distribution misalignment. Circumventing retrieval, our analysis shows this approach offers rapid answer generation to almost any language off-the-shelf, without the need for any additional training data in the target language.
Experiential Learning Approach for Software Engineering Courses at Higher Education Level
Venue: arXiv cs.SE 2021
Year: 2021

Background: Software project management activities help to introduce software process models in Software Engineering courses. However, these activities should be adequately aligned with the learning outcomes and support student's progression. Objective: Present and evaluate an approach to help students acquire theoretical and practical knowledge and experience real-world software projects' challenges. The approach combines a serious game and a design-implement task in which students develop a controlled-scale software system. Method: To evaluate our approach, we analyzed the students' perceptions collected through an online survey, their project plans, and their final reports using thematic analysis. Results: Results suggest that the approach promotes knowledge acquisition, enables students' progression, reinforces theoretical concepts, and is properly aligned with the course's learning outcomes. Conclusion: The approach seems to help to introduce software process models in Software Engineering courses. Our experience can also be inspiring for educators willing to apply our approach in similar courses.
Longitudinal diffusion MRI analysis using Segis-Net: a single-step deep-learning framework for simultaneous segmentation and registration
Venue: arXiv cs.CV 2021
Year: 2021

This work presents a single-step deep-learning framework for longitudinal image analysis, coined Segis-Net. To optimally exploit information available in longitudinal data, this method concurrently learns a multi-class segmentation and nonlinear registration. Segmentation and registration are modeled using a convolutional neural network and optimized simultaneously for their mutual benefit. An objective function that optimizes spatial correspondence for the segmented structures across time-points is proposed. We applied Segis-Net to the analysis of white matter tracts from N=8045 longitudinal brain MRI datasets of 3249 elderly individuals. Segis-Net approach showed a significant increase in registration accuracy, spatio-temporal segmentation consistency, and reproducibility comparing with two multistage pipelines. This also led to a significant reduction in the sample-size that would be required to achieve the same statistical power in analyzing tract-specific measures. Thus, we expect that Segis-Net can serve as a new reliable tool to support longitudinal imaging studies to investigate macro- and microstructural brain changes over time.
Adaptive Threshold for Online Object Recognition and Re-identification Tasks
Venue: arXiv cs.CV 2021
Year: 2021

Choosing a decision threshold is one of the challenging job in any classification tasks. How much the model is accurate, if the deciding boundary is not picked up carefully, its entire performance would go in vain. On the other hand, for imbalance classification where one of the classes is dominant over another, relying on the conventional method of choosing threshold would result in poor performance. Even if the threshold or decision boundary is properly chosen based on machine learning strategies like SVM and decision tree, it will fail at some point for dynamically varying databases and in case of identity-features that are more or less similar, like in face recognition and person re-identification models. Hence, with the need for adaptability of the decision threshold selection for imbalanced classification and incremental database size, an online optimization-based statistical feature learning adaptive technique is developed and tested on the LFW datasets and self-prepared athletes datasets. This method of adopting adaptive threshold resulted in 12-45% improvement in the model accuracy compared to the fixed threshold {0.3,0.5,0.7} that are usually taken via the hit-and-trial method in any classification and identification tasks. Source code for the complete algorithm is available at: https://github.com/Varat7v2/adaptive-threshold
Local-to-Global Contraction in Simplicial Complexes
Venue: arXiv cs.DS 2021
Year: 2021

We give a local-to-global principle for relative entropy contraction in simplicial complexes. This is similar to the local-to-global principle for variances obtained by Alev and Lau (2020).
TEMPI: An Interposed MPI Library with a Canonical Representation of CUDA-aware Datatypes
Venue: arXiv cs.DC 2021
Year: 2021

MPI derived datatypes are an abstraction that simplifies handling of non-contiguous data in MPI applications. These datatypes are recursively constructed at runtime from primitive Named Types defined in the MPI standard. More recently, the development and deployment of CUDA-aware MPI implementations has encouraged the transition of distributed high-performance MPI codes to use GPUs. Such implementations allow MPI functions to directly operate on GPU buffers, easing integration of GPU compute into MPI codes. This work first presents a novel datatype handling strategy for nested strided datatypes, which finds a middle ground between the specialized or generic handling in prior work. This work also shows that the performance characteristics of non-contiguous data handling can be modeled with empirical system measurements, and used to transparently improve MPI_Send/Recv latency. Finally, despite substantial attention to non-contiguous GPU data and CUDA-aware MPI implementations, good performance cannot be taken for granted. This work demonstrates its contributions through an MPI interposer library, TEMPI. TEMPI can be used with existing MPI deployments without system or application changes. Ultimately, the interposed-library model of this work demonstrates MPI_Pack speedup of up to 242000x and MPI_Send speedup of up to 59000x compared to the MPI implementation deployed on a leadership-class supercomputer. This yields speedup of more than 917x in a 3D halo exchange with 3072 processes.
Causal Perception in Question-Answering Systems
Venue: arXiv cs.HC 2021
Year: 2021

Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impact of showing different information on user perceptions of a question-answering system. We found that in a system that occasionally provided unreasonable responses, showing a scatterplot increased the plausibility of unreasonable causal claims. Also, simply warning participants that correlation is not causation seemed to lead participants to accept reasonable causal claims more cautiously. We observed a strong tendency among participants to associate correlation with causation. Yet, the warning appeared to reduce the tendency. Grounded in the findings, we propose ways to reduce the illusion of causality when using question-answering systems.
Misplaced Subsequences Repairing with Application to Multivariate Industrial Time Series Data
Venue: arXiv cs.DB 2021
Year: 2021

Both the volume and the collection velocity of time series generated by monitoring sensors are increasing in the Internet of Things (IoT). Data management and analysis requires high quality and applicability of the IoT data. However, errors are prevalent in original time series data. Inconsistency in time series is a serious data quality problem existing widely in IoT. Such problem could be hardly solved by existing techniques. Motivated by this, we define an inconsistent subsequences problem in multivariate time series, and propose an integrity data repair approach to solve inconsistent problems. Our proposed repairing method consists of two parts: (1) we design effective anomaly detection method to discover latent inconsistent subsequences in the IoT time series; and (2) we develop repair algorithms to precisely locate the start and finish time of inconsistent intervals, and provide reliable repairing strategies. A thorough experiment on two real-life datasets verifies the superiority of our method compared to other practical approaches. Experimental results also show that our method captures and repairs inconsistency problems effectively in industrial time series in complex IIoT scenarios.
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade
Venue: arXiv cs.CL 2021
Year: 2021

Dynamic early exiting aims to accelerate the inference of pre-trained language models (PLMs) by emitting predictions in internal layers without passing through the entire model. In this paper, we empirically analyze the working mechanism of dynamic early exiting and find that it faces a performance bottleneck under high speed-up ratios. On one hand, the PLMs' representations in shallow layers lack high-level semantic information and thus are not sufficient for accurate predictions. On the other hand, the exiting decisions made by internal classifiers are unreliable, leading to wrongly emitted early predictions. We instead propose a new framework for accelerating the inference of PLMs, CascadeBERT, which dynamically selects proper-sized and complete models in a cascading manner, providing comprehensive representations for predictions. We further devise a difficulty-aware objective, encouraging the model to output the class probability that reflects the real difficulty of each instance for a more reliable cascading mechanism. Experimental results show that CascadeBERT can achieve an overall 15\% improvement under 4$\times$ speed-up compared with existing dynamic early exiting methods on six classification tasks, yielding more calibrated and accurate predictions.
Combining Semilattices and Semimodules
Venue: arXiv cs.CL 2021
Year: 2021

We describe the canonical weak distributive law $δ\colon \mathcal S \mathcal P \to \mathcal P \mathcal S$ of the powerset monad $\mathcal P$ over the $S$-left-semimodule monad $\mathcal S$, for a class of semirings $S$. We show that the composition of $\mathcal P$ with $\mathcal S$ by means of such $δ$ yields almost the monad of convex subsets previously introduced by Jacobs: the only difference consists in the absence in Jacobs's monad of the empty convex set. We provide a handy characterisation of the canonical weak lifting of $\mathcal P$ to $\mathbb{EM}(\mathcal S)$ as well as an algebraic theory for the resulting composed monad. Finally, we restrict the composed monad to finitely generated convex subsets and we show that it is presented by an algebraic theory combining semimodules and semilattices with bottom, which are the algebras for the finite powerset monad $\mathcal P_f$.
Dialogue Graph Modeling for Conversational Machine Reading
Venue: arXiv cs.CL 2021
Year: 2021

Conversational Machine Reading (CMR) aims at answering questions in a complicated manner. Machine needs to answer questions through interactions with users based on given rule document, user scenario and dialogue history, and ask questions to clarify if necessary. In this paper, we propose a dialogue graph modeling framework to improve the understanding and reasoning ability of machine on CMR task. There are three types of graph in total. Specifically, Discourse Graph is designed to learn explicitly and extract the discourse relation among rule texts as well as the extra knowledge of scenario; Decoupling Graph is used for understanding local and contextualized connection within rule texts. And finally a global graph for fusing the information together and reply to the user with our final decision being either "Yes/No/Irrelevant" or to ask a follow-up question to clarify.
WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections
Venue: arXiv cs.CL 2021
Year: 2021

Datasets for data-to-text generation typically focus either on multi-domain, single-sentence generation or on single-domain, long-form generation. In this work, we cast generating Wikipedia sections as a data-to-text generation task and create a large-scale dataset, WikiTableT, that pairs Wikipedia sections with their corresponding tabular data and various metadata. WikiTableT contains millions of instances, covering a broad range of topics, as well as a variety of flavors of generation tasks with different levels of flexibility. We benchmark several training and decoding strategies on WikiTableT. Our qualitative analysis shows that the best approaches can generate fluent and high quality texts but they struggle with coherence and factuality, showing the potential for our dataset to inspire future work on long-form generation.
Consensus-Based Distributed Computation of Link-Based Network Metrics
Venue: arXiv cs.DC 2021
Year: 2021

Average consensus algorithms have wide applications in distributed computing systems where all the nodes agree on the average value of their initial states by only exchanging information with their local neighbors. In this letter, we look into link-based network metrics which are polynomial functions of pair-wise node attributes defined over the links in a network. Different from node-based average consensus, such link-based metrics depend on both the distribution of node attributes and the underlying network topology. We propose a general algorithm using the weighted average consensus protocol for the distributed computation of link-based network metrics and provide the convergence conditions and convergence rate analysis.