OpenRaiser/Intern-Atlas huggingface dataset:
Cheap and Good? Simple and Effective Data Augmentation for Low Resource Machine Reading
Venue: arXiv cs.CL 2021
Year: 2021

We propose a simple and effective strategy for data augmentation for low-resource machine reading comprehension (MRC). Our approach first pretrains the answer extraction components of a MRC system on the augmented data that contains approximate context of the correct answers, before training it on the exact answer spans. The approximate context helps the QA method components in narrowing the location of the answers. We demonstrate that our simple strategy substantially improves both document retrieval and answer extraction performance by providing larger context of the answers and additional training data. In particular, our method significantly improves the performance of BERT based retriever (15.12\%), and answer extractor (4.33\% F1) on TechQA, a complex, low-resource MRC task. Further, our data augmentation strategy yields significant improvements of up to 3.9\% exact match (EM) and 2.7\% F1 for answer extraction on PolicyQA, another practical but moderate sized QA dataset that also contains long answer spans.
RECOWNs: Probabilistic Circuits for Trustworthy Time Series Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

Time series forecasting is a relevant task that is performed in several real-world scenarios such as product sales analysis and prediction of energy demand. Given their accuracy performance, currently, Recurrent Neural Networks (RNNs) are the models of choice for this task. Despite their success in time series forecasting, less attention has been paid to make the RNNs trustworthy. For example, RNNs can not naturally provide an uncertainty measure to their predictions. This could be extremely useful in practice in several cases e.g. to detect when a prediction might be completely wrong due to an unusual pattern in the time series. Whittle Sum-Product Networks (WSPNs), prominent deep tractable probabilistic circuits (PCs) for time series, can assist an RNN with providing meaningful probabilities as uncertainty measure. With this aim, we propose RECOWN, a novel architecture that employs RNNs and a discriminant variant of WSPNs called Conditional WSPNs (CWSPNs). We also formulate a Log-Likelihood Ratio Score as better estimation of uncertainty that is tailored to time series and Whittle likelihoods. In our experiments, we show that RECOWNs are accurate and trustworthy time series predictors, able to "know when they do not know".
Few-Shot Action Localization without Knowing Boundaries
Venue: arXiv cs.CV 2021
Year: 2021

Learning to localize actions in long, cluttered, and untrimmed videos is a hard task, that in the literature has typically been addressed assuming the availability of large amounts of annotated training samples for each class -- either in a fully-supervised setting, where action boundaries are known, or in a weakly-supervised setting, where only class labels are known for each video. In this paper, we go a step further and show that it is possible to learn to localize actions in untrimmed videos when a) only one/few trimmed examples of the target action are available at test time, and b) when a large collection of videos with only class label annotation (some trimmed and some weakly annotated untrimmed ones) are available for training; with no overlap between the classes used during training and testing. To do so, we propose a network that learns to estimate Temporal Similarity Matrices (TSMs) that model a fine-grained similarity pattern between pairs of videos (trimmed or untrimmed), and uses them to generate Temporal Class Activation Maps (TCAMs) for seen or unseen classes. The TCAMs serve as temporal attention mechanisms to extract video-level representations of untrimmed videos, and to temporally localize actions at test time. To the best of our knowledge, we are the first to propose a weakly-supervised, one/few-shot action localization network that can be trained in an end-to-end fashion. Experimental results on THUMOS14 and ActivityNet1.2 datasets, show that our method achieves performance comparable or better to state-of-the-art fully-supervised, few-shot learning methods.
FPT Algorithms to Compute the Elimination Distance to Bipartite Graphs and More
Venue: arXiv cs.DS 2021
Year: 2021

For a hereditary graph class $\mathcal{H}$, the $\mathcal{H}$-elimination distance of a graph $G$ is the minimum number of rounds needed to reduce $G$ to a member of $\mathcal{H}$ by removing one vertex from each connected component in each round. The $\mathcal{H}$-treewidth of a graph $G$ is the minimum, taken over all vertex sets $X$ for which each connected component of $G - X$ belongs to $\mathcal{H}$, of the treewidth of the graph obtained from $G$ by replacing the neighborhood of each component of $G-X$ by a clique and then removing $V(G) \setminus X$. These parameterizations recently attracted interest because they are simultaneously smaller than the graph-complexity measures treedepth and treewidth, respectively, and the vertex-deletion distance to $\mathcal{H}$. For the class $\mathcal{H}$ of bipartite graphs, we present non-uniform fixed-parameter tractable algorithms for testing whether the $\mathcal{H}$-elimination distance or $\mathcal{H}$-treewidth of a graph is at most $k$. Along the way, we also provide such algorithms for all graph classes $\mathcal{H}$ defined by a finite set of forbidden induced subgraphs.
Realistic Evaluation Principles for Cross-document Coreference Resolution
Venue: arXiv cs.CL 2021
Year: 2021

We point out that common evaluation practices for cross-document coreference resolution have been unrealistically permissive in their assumed settings, yielding inflated results. We propose addressing this issue via two evaluation methodology principles. First, as in other tasks, models should be evaluated on predicted mentions rather than on gold mentions. Doing this raises a subtle issue regarding singleton coreference clusters, which we address by decoupling the evaluation of mention detection from that of coreference linking. Second, we argue that models should not exploit the synthetic topic structure of the standard ECB+ dataset, forcing models to confront the lexical ambiguity challenge, as intended by the dataset creators. We demonstrate empirically the drastic impact of our more realistic evaluation principles on a competitive model, yielding a score which is 33 F1 lower compared to evaluating by prior lenient practices.
Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces
Venue: arXiv cs.RO 2021
Year: 2021

Decision making under uncertainty can be framed as a partially observable Markov decision process (POMDP). Finding exact solutions of POMDPs is generally computationally intractable, but the solution can be approximated by sampling-based approaches. These sampling-based POMDP solvers rely on multi-armed bandit (MAB) heuristics, which assume the outcomes of different actions to be uncorrelated. In some applications, like motion planning in continuous spaces, similar actions yield similar outcomes. In this paper, we utilize variants of MAB heuristics that make Lipschitz continuity assumptions on the outcomes of actions to improve the efficiency of sampling-based planning approaches. We demonstrate the effectiveness of this approach in the context of motion planning for automated driving.
Defining definition: a Text mining Approach to Define Innovative Technological Fields
Venue: arXiv cs.IR 2021
Year: 2021

One of the first task of an innovative project is delineating the scope of the project itself or of the product/service to be developed. A wrong scope definition can determine (in the worst case) project failure. A good scope definition become even more relevant in technological intensive innovation projects, nowadays characterized by a highly dynamic multidisciplinary, turbulent and uncertain environment. In these cases, the boundaries of the project are not easily detectable and it is difficult to decide what it is in-scope and out-of-scope. The present work proposes a tool for the scope delineation process, that automatically define an innovative technological field or a new technology. The tool is based on Text Mining algorithm that exploits Elsevier's Scopus abstracts in order to the extract relevant data to define a technological scope. The automatic definition tool is then applied on four case studies: Artificial Intelligence and Data Science. The results show how the tool can provide many crucial information in the definition process of a technological field. In particular for the target technological field (or technology), it provides the definition and other elements related to the target.
A Falta de Pan, Buenas Son Tortas: The Efficacy of Predicted UPOS Tags for Low Resource UD Parsing
Venue: arXiv cs.CL 2021
Year: 2021

We evaluate the efficacy of predicted UPOS tags as input features for dependency parsers in lower resource settings to evaluate how treebank size affects the impact tagging accuracy has on parsing performance. We do this for real low resource universal dependency treebanks, artificially low resource data with varying treebank sizes, and for very small treebanks with varying amounts of augmented data. We find that predicted UPOS tags are somewhat helpful for low resource treebanks, especially when fewer fully-annotated trees are available. We also find that this positive impact diminishes as the amount of data increases.
Supervised Machine Learning with Plausible Deniability
Venue: arXiv cs.LG 2021
Year: 2021

We study the question of how well machine learning (ML) models trained on a certain data set provide privacy for the training data, or equivalently, whether it is possible to reverse-engineer the training data from a given ML model. While this is easy to answer negatively in the most general case, it is interesting to note that the protection extends over non-recoverability towards plausible deniability: Given an ML model $f$, we show that one can take a set of purely random training data, and from this define a suitable ``learning rule'' that will produce a ML model that is exactly $f$. Thus, any speculation about which data has been used to train $f$ is deniable upon the claim that any other data could have led to the same results. We corroborate our theoretical finding with practical examples, and open source implementations of how to find the learning rules for a chosen set of raining data.
Staircase Attention for Recurrent Processing of Sequences
Venue: arXiv cs.LG 2021
Year: 2021

Attention mechanisms have become a standard tool for sequence modeling tasks, in particular by stacking self-attention layers over the entire input sequence as in the Transformer architecture. In this work we introduce a novel attention procedure called staircase attention that, unlike self-attention, operates across the sequence (in time) recurrently processing the input by adding another step of processing. A step in the staircase comprises of backward tokens (encoding the sequence so far seen) and forward tokens (ingesting a new part of the sequence), or an extreme Ladder version with a forward step of zero that simply repeats the Transformer on each step of the ladder, sharing the weights. We thus describe a family of such models that can trade off performance and compute, by either increasing the amount of recurrence through time, the amount of sequential processing via recurrence in depth, or both. Staircase attention is shown to be able to solve tasks that involve tracking that conventional Transformers cannot, due to this recurrence. Further, it is shown to provide improved modeling power for the same size model (number of parameters) compared to self-attentive Transformers on large language modeling and dialogue tasks, yielding significant perplexity gains.
Optimizing a Binary Intelligent Reflecting Surface for OFDM Communications under Mutual Coupling
Venue: arXiv cs.IT 2021
Year: 2021

An intelligent reflecting surface (IRS) can greatly improve the channel quality over a frequency-flat channel, if it is configured to reflect the incident signal as a beam towards the receiver. However, the fundamental limitations of the IRS technology become apparent over practical frequency-selective channels, where the same configuration must be used over the entire bandwidth. In this paper, we consider a wideband orthogonal frequency-division multiplexing (OFDM) system that is supported by a fairly realistic IRS setup with two unbalanced states per element and also mutual coupling. We describe the simulation setup considered in the IEEE Signal Processing Cup 2021, propose a low-complexity solution for channel estimation and IRS configuration, and evaluate it on that setup.
Modeling Uplink Coverage Performance in Hybrid Satellite-Terrestrial Networks
Venue: arXiv cs.IT 2021
Year: 2021

Once deemed a far-fetched vision, emerging deployments of massive satellite constellations will soon offer true global coverage. When these constellations overlay the evolving terrestrial networks, a new hybrid continuum is formed which has the potential to provide uninterrupted coverage. In this paper, we provide an analytic framework for the uplink coverage probability in hybrid satellite-terrestrial networks. The framework extends well-developed terrestrial models that utilize tools from stochastic geometry by incorporating an additional layer that fits the emerging next generation satellite-terrestrial networks. The paper captures the impact of both (i) the constellation size, and (ii) the terrestrial base station density on the coverage of the uplink traffic which is dominant in applications relying on wireless sensor networks, such as the Internet of Things. This framework provides insights that guide the design of hybrid network infrastructures for a desired quality of service.
RobustBF: A High Accuracy and Memory Efficient 2D Bloom Filter
Venue: arXiv cs.DS 2021
Year: 2021

Bloom Filter is an important probabilistic data structure to reduce memory consumption for membership filters. It is applied in diverse domains such as Computer Networking, Network Security and Privacy, IoT, Edge Computing, Cloud Computing, Big Data, and Biometrics. But Bloom Filter has an issue of the false positive probability. To address this issue, we propose a novel robust Bloom Filter, robustBF for short. robustBF is a 2D Bloom Filter, capable of filtering millions of data with high accuracy without compromising the performance. Our proposed system is presented in two-fold. Firstly, we modify the murmur hash function, and test all modified hash functions for improvements and select the best-modified hash function experimentally. Secondly, we embed the modified hash functions in 2D Bloom Filter. Our experimental results show that robustBF is better than standard Bloom Filter and counting Bloom Filter in every aspect. robustBF exhibits nearly zero false positive probability with more than $10\times$ and $44\times$ lower memory consumption than standard Bloom filter and counting Bloom Filter, respectively.
Property-Aware Robot Object Manipulation: a Generative Approach
Venue: arXiv cs.RO 2021
Year: 2021

When transporting an object, we unconsciously adapt our movement to its properties, for instance by slowing down when the item is fragile. The most relevant features of an object are immediately revealed to a human observer by the way the handling occurs, without any need for verbal description. It would greatly facilitate collaboration to enable humanoid robots to perform movements that convey similar intuitive cues to the observers. In this work, we focus on how to generate robot motion adapted to the hidden properties of the manipulated objects, such as their weight and fragility. We explore the possibility of leveraging Generative Adversarial Networks to synthesize new actions coherent with the properties of the object. The use of a generative approach allows us to create new and consistent motion patterns, without the need of collecting a large number of recorded human-led demonstrations. Besides, the informative content of the actions is preserved. Our results show that Generative Adversarial Nets can be a powerful tool for the generation of novel and meaningful transportation actions, which result effectively modulated as a function of the object weight and the carefulness required in its handling.
Noise Conditional Flow Model for Learning the Super-Resolution Space
Venue: arXiv cs.CV 2021
Year: 2021

Fundamentally, super-resolution is ill-posed problem because a low-resolution image can be obtained from many high-resolution images. Recent studies for super-resolution cannot create diverse super-resolution images. Although SRFlow tried to account for ill-posed nature of the super-resolution by predicting multiple high-resolution images given a low-resolution image, there is room to improve the diversity and visual quality. In this paper, we propose Noise Conditional flow model for Super-Resolution, NCSR, which increases the visual quality and diversity of images through noise conditional layer. To learn more diverse data distribution, we add noise to training data. However, low-quality images are resulted from adding noise. We propose the noise conditional layer to overcome this phenomenon. The noise conditional layer makes our model generate more diverse images with higher visual quality than other works. Furthermore, we show that this layer can overcome data distribution mismatch, a problem that arises in normalizing flow models. With these benefits, NCSR outperforms baseline in diversity and visual quality and achieves better visual quality than traditional GAN-based models. We also get outperformed scores at NTIRE 2021 challenge.
CLTR: An End-to-End, Transformer-Based System for Cell Level Table Retrieval and Table Question Answering
Venue: arXiv cs.CL 2021
Year: 2021

We present the first end-to-end, transformer-based table question answering (QA) system that takes natural language questions and massive table corpus as inputs to retrieve the most relevant tables and locate the correct table cells to answer the question. Our system, CLTR, extends the current state-of-the-art QA over tables model to build an end-to-end table QA architecture. This system has successfully tackled many real-world table QA problems with a simple, unified pipeline. Our proposed system can also generate a heatmap of candidate columns and rows over complex tables and allow users to quickly identify the correct cells to answer questions. In addition, we introduce two new open-domain benchmarks, E2E_WTQ and E2E_GNQ, consisting of 2,005 natural language questions over 76,242 tables. The benchmarks are designed to validate CLTR as well as accommodate future table retrieval and end-to-end table QA research and experiments. Our experiments demonstrate that our system is the current state-of-the-art model on the table retrieval task and produces promising results for end-to-end table QA.
Computation Offloading at Field Level: Motivation and Break-Even Point Calculation
Venue: arXiv cs.NI 2021
Year: 2021

Smart manufacturing has the objective of creating highly flexible and resource optimized industrial plants. Furthermore, the improvement of product quality is another important target. These requirements implicate more complex control algo-rithms. Processing these algorithms may exceed the capabilities of resource constrained devices, such as programmable logic controllers (PLCs). In this case, the necessity for computation offloading is given. Due to the fact that industrial plants are currently designed for a life-cycle-time of more than ten years, in a realistic smart manufacturing scenario, these devices have to be considered. Therefore, we investigate the impact of complex algorithms on conventional PLCs by simulating them with a load generator. In addition, we propose a realistic factory scenario including benchmarks for both wireline and wireless communication systems. Thus, their round-trip time (RTT) is measured with and without additional load on the network. With the help of these investigations, break-even points for the application of computation offloading of two typical PLCs of Siemens S7 series can be calculated.
Interpreting Deep Learning based Cerebral Palsy Prediction with Channel Attention
Venue: arXiv cs.CV 2021
Year: 2021

Early prediction of cerebral palsy is essential as it leads to early treatment and monitoring. Deep learning has shown promising results in biomedical engineering thanks to its capacity of modelling complicated data with its non-linear architecture. However, due to their complex structure, deep learning models are generally not interpretable by humans, making it difficult for clinicians to rely on the findings. In this paper, we propose a channel attention module for deep learning models to predict cerebral palsy from infants' body movements, which highlights the key features (i.e. body joints) the model identifies as important, thereby indicating why certain diagnostic results are found. To highlight the capacity of the deep network in modelling input features, we utilize raw joint positions instead of hand-crafted features. We validate our system with a real-world infant movement dataset. Our proposed channel attention module enables the visualization of the vital joints to this disease that the network considers. Our system achieves 91.67% accuracy, suppressing other state-of-the-art deep learning methods.
A Deep Value-network Based Approach for Multi-Driver Order Dispatching
Venue: arXiv cs.LG 2021
Year: 2021

Recent works on ride-sharing order dispatching have highlighted the importance of taking into account both the spatial and temporal dynamics in the dispatching process for improving the transportation system efficiency. At the same time, deep reinforcement learning has advanced to the point where it achieves superhuman performance in a number of fields. In this work, we propose a deep reinforcement learning based solution for order dispatching and we conduct large scale online A/B tests on DiDi's ride-dispatching platform to show that the proposed method achieves significant improvement on both total driver income and user experience related metrics. In particular, we model the ride dispatching problem as a Semi Markov Decision Process to account for the temporal aspect of the dispatching actions. To improve the stability of the value iteration with nonlinear function approximators like neural networks, we propose Cerebellar Value Networks (CVNet) with a novel distributed state representation layer. We further derive a regularized policy evaluation scheme for CVNet that penalizes large Lipschitz constant of the value network for additional robustness against adversarial perturbation and noises. Finally, we adapt various transfer learning methods to CVNet for increased learning adaptability and efficiency across multiple cities. We conduct extensive offline simulations based on real dispatching data as well as online AB tests through the DiDi's platform. Results show that CVNet consistently outperforms other recently proposed dispatching methods. We finally show that the performance can be further improved through the efficient use of transfer learning.