OpenRaiser/Intern-Atlas huggingface dataset:
Optimizing Edge Detection for Image Segmentation with Multicut Penalties
Venue: arXiv cs.CV 2021
Year: 2021

The Minimum Cost Multicut Problem (MP) is a popular way for obtaining a graph decomposition by optimizing binary edge labels over edge costs. While the formulation of a MP from independently estimated costs per edge is highly flexible and intuitive, solving the MP is NP-hard and time-expensive. As a remedy, recent work proposed to predict edge probabilities with awareness to potential conflicts by incorporating cycle constraints in the prediction process. We argue that such formulation, while providing a first step towards end-to-end learnable edge weights, is suboptimal, since it is built upon a loose relaxation of the MP. We therefore propose an adaptive CRF that allows to progressively consider more violated constraints and, in consequence, to issue solutions with higher validity. Experiments on the BSDS500 benchmark for natural image segmentation as well as on electron microscopic recordings show that our approach yields more precise edge detection and image segmentation.
Couplformer:Rethinking Vision Transformer with Coupling Attention Map
Venue: arXiv cs.CV 2021
Year: 2021

With the development of the self-attention mechanism, the Transformer model has demonstrated its outstanding performance in the computer vision domain. However, the massive computation brought from the full attention mechanism became a heavy burden for memory consumption. Sequentially, the limitation of memory reduces the possibility of improving the Transformer model. To remedy this problem, we propose a novel memory economy attention mechanism named Couplformer, which decouples the attention map into two sub-matrices and generates the alignment scores from spatial information. A series of different scale image classification tasks are applied to evaluate the effectiveness of our model. The result of experiments shows that on the ImageNet-1k classification task, the Couplformer can significantly decrease 28% memory consumption compared with regular Transformer while accessing sufficient accuracy requirements and outperforming 0.92% on Top-1 accuracy while occupying the same memory footprint. As a result, the Couplformer can serve as an efficient backbone in visual tasks, and provide a novel perspective on the attention mechanism for researchers.
A Reinforcement Learning-based Adaptive Control Model for Future Street Planning, An Algorithm and A Case Study
Venue: arXiv cs.AI 2021
Year: 2021

With the emerging technologies in Intelligent Transportation System (ITS), the adaptive operation of road space is likely to be realised within decades. An intelligent street can learn and improve its decision-making on the right-of-way (ROW) for road users, liberating more active pedestrian space while maintaining traffic safety and efficiency. However, there is a lack of effective controlling techniques for these adaptive street infrastructures. To fill this gap in existing studies, we formulate this control problem as a Markov Game and develop a solution based on the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The proposed model can dynamically assign ROW for sidewalks, autonomous vehicles (AVs) driving lanes and on-street parking areas in real-time. Integrated with the SUMO traffic simulator, this model was evaluated using the road network of the South Kensington District against three cases of divergent traffic conditions: pedestrian flow rates, AVs traffic flow rates and parking demands. Results reveal that our model can achieve an average reduction of 3.87% and 6.26% in street space assigned for on-street parking and vehicular operations. Combined with space gained by limiting the number of driving lanes, the average proportion of sidewalks to total widths of streets can significantly increase by 10.13%.
DEBACER: a method for slicing moderated debates
Venue: arXiv cs.AI 2021
Year: 2021

Subjects change frequently in moderated debates with several participants, such as in parliamentary sessions, electoral debates, and trials. Partitioning a debate into blocks with the same subject is essential for understanding. Often a moderator is responsible for defining when a new block begins so that the task of automatically partitioning a moderated debate can focus solely on the moderator's behavior. In this paper, we (i) propose a new algorithm, DEBACER, which partitions moderated debates; (ii) carry out a comparative study between conventional and BERTimbau pipelines; and (iii) validate DEBACER applying it to the minutes of the Assembly of the Republic of Portugal. Our results show the effectiveness of DEBACER. Keywords: Natural Language Processing, Political Documents, Spoken Text Processing, Speech Split, Dialogue Partitioning.
Improving the Question Answering Quality using Answer Candidate Filtering based on Natural-Language Features
Venue: arXiv cs.IR 2021
Year: 2021

Software with natural-language user interfaces has an ever-increasing importance. However, the quality of the included Question Answering (QA) functionality is still not sufficient regarding the number of questions that are answered correctly. In our work, we address the research problem of how the QA quality of a given system can be improved just by evaluating the natural-language input (i.e., the user's question) and output (i.e., the system's answer). Our main contribution is an approach capable of identifying wrong answers provided by a QA system. Hence, filtering incorrect answers from a list of answer candidates is leading to a highly improved QA quality. In particular, our approach has shown its potential while removing in many cases the majority of incorrect answers, which increases the QA quality significantly in comparison to the non-filtered output of a system.
Network Compression via Central Filter
Venue: arXiv cs.LG 2021
Year: 2021

Neural network pruning has remarkable performance for reducing the complexity of deep network models. Recent network pruning methods usually focused on removing unimportant or redundant filters in the network. In this paper, by exploring the similarities between feature maps, we propose a novel filter pruning method, Central Filter (CF), which suggests that a filter is approximately equal to a set of other filters after appropriate adjustments. Our method is based on the discovery that the average similarity between feature maps changes very little, regardless of the number of input images. Based on this finding, we establish similarity graphs on feature maps and calculate the closeness centrality of each node to select the Central Filter. Moreover, we design a method to directly adjust weights in the next layer corresponding to the Central Filter, effectively minimizing the error caused by pruning. Through experiments on various benchmark networks and datasets, CF yields state-of-the-art performance. For example, with ResNet-56, CF reduces approximately 39.7% of FLOPs by removing 47.1% of the parameters, with even 0.33% accuracy improvement on CIFAR-10. With GoogLeNet, CF reduces approximately 63.2% of FLOPs by removing 55.6% of the parameters, with only a small loss of 0.35% in top-1 accuracy on CIFAR-10. With ResNet-50, CF reduces approximately 47.9% of FLOPs by removing 36.9% of the parameters, with only a small loss of 1.07% in top-1 accuracy on ImageNet. The codes can be available at https://github.com/8ubpshLR23/Central-Filter.
Combining Design Thinking and Software Requirements Engineering to create Human-centered Software-intensive Systems
Venue: arXiv cs.SE 2021
Year: 2021

Effective Requirements Engineering is a crucial activity in softwareintensive development projects. The human-centric working mode of Design Thinking is considered a powerful way to complement such activities when designing innovative systems. Research has already made great strides to illustrate the benefits of using Design Thinking for Requirements Engineering. However, it has remained mostly unclear how to actually realize a combination of both. In this chapter, we contribute an artifact-based model that integrates Design Thinking and Requirements Engineering for innovative software-intensive systems. Drawing from our research and project experiences, we suggest three strategies for tailoring and integrating Design Thinking and Requirements Engineering with complementary synergies.
Paradigms of Computational Agency
Venue: arXiv cs.AI 2021
Year: 2021

Agent-based models have emerged as a promising paradigm for addressing ever increasing complexity of information systems. In its initial days in the 1990s when object-oriented modeling was at its peak, an agent was treated as a special kind of "object" that had a persistent state and its own independent thread of execution. Since then, agent-based models have diversified enormously to even open new conceptual insights about the nature of systems in general. This paper presents a perspective on the disparate ways in which our understanding of agency, as well as computational models of agency have evolved. Advances in hardware like GPUs, that brought neural networks back to life, may also similarly infuse new life into agent-based models, as well as pave the way for advancements in research on Artificial General Intelligence (AGI).
GPU-accelerated image alignment for object detection in industrial applications
Venue: arXiv cs.CV 2021
Year: 2021

This research proposes a practical method for detecting featureless objects by using image alignment approach with a robust similarity measure in industrial applications. This similarity measure is robust against occlusion, illumination changes and background clutter. The performance of the proposed GPU (Graphics Processing Unit) accelerated algorithm is deemed successful in experiments of comparison between both CPU and GPU implementations
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation
Venue: arXiv cs.CV 2021
Year: 2021

Most existing vision-language pre-training methods focus on understanding tasks and use BERT-like objectives (masked language modeling and image-text matching) during pretraining. Although they perform well in many understanding downstream tasks, e.g., visual question answering, image-text retrieval and visual entailment, they do not possess the ability to generate. To tackle this problem, we propose Unified multimodal pre-training for both Vision-Language understanding and generation (UniVL). The proposed UniVL is capable of handling both understanding tasks and generative tasks. We augment existing pretraining paradigms that only use random masks with causal masks, i.e., triangular masks that mask out future tokens, such that the pre-trained models can have autoregressive generation abilities by design. We formulate several previous understanding tasks as a text generation task and propose to use prompt-based method for fine-tuning on different downstream tasks. Our experiments show that there is a trade-off between understanding tasks and generation tasks while using the same model, and a feasible way to improve both tasks is to use more data. Our UniVL framework attains comparable performance to recent vision-language pre-training methods on both understanding tasks and generation tasks. Moreover, we demostrate that prompt-based finetuning is more data-efficient - it outperforms discriminative methods in few-shot scenarios.
Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models
Venue: arXiv cs.CR 2021
Year: 2021

Deep learning (DL) models, especially those large-scale and high-performance ones, can be very costly to train, demanding a great amount of data and computational resources. Unauthorized reproduction of DL models can lead to copyright infringement and cause huge economic losses to model owners. Existing copyright protection techniques are mostly based on watermarking, which embeds an owner-specified watermark into the model. While being able to provide exact ownership verification, these techniques are 1) invasive, as they need to tamper with the training process, which may affect the utility or introduce new security risks; 2) prone to adaptive attacks that attempt to remove the watermark; and 3) not robust to the emerging model extraction attacks. Latest fingerprinting work, though being non-invasive, also falls short when facing the diverse and ever-growing attack scenarios. In this paper, we propose a novel testing framework for DL copyright protection: DEEPJUDGE. DEEPJUDGE quantitatively tests the similarities between two DL models: a victim model and a suspect model. It leverages a diverse set of testing metrics and test case generation methods to produce a chain of supporting evidence to help determine whether a suspect model is a copy of the victim model. Advantages of DEEPJUDGE include: 1) non-invasive, as it works directly on the model and does not tamper with the training process; 2) efficient, as it only needs a small set of test cases and a quick scan of models; 3) flexible, as it can easily incorporate new metrics or generation methods to obtain more confident judgement; and 4) fairly robust to model extraction and adaptive attacks. We verify the effectiveness of DEEPJUDGE under typical copyright infringement scenarios, including model finetuning, pruning and extraction, via extensive experiments on both image and speech datasets with a variety of model architectures.
Singleton-type bounds for list-decoding and list-recovery, and related results
Venue: arXiv cs.IT 2021
Year: 2021

List-decoding and list-recovery are important generalizations of unique decoding that received considerable attention over the years. However, the optimal trade-off among list-decoding (resp. list-recovery) radius, list size, and the code rate are not fully understood in both problems. This paper takes a step towards this direction when the list size is a given constant and the alphabet size is large (as a function of the code length). We prove a new Singleton-type upper bound for list-decodable codes, which improves upon the previously known bound by roughly a factor of $1/L$, where $L$ is the list size. We also prove a Singleton-type upper bound for list-recoverable codes, which is to the best of our knowledge, the first such bound for list-recovery. We apply these results to obtain new lower bounds that are optimal up to a multiplicative constant on the list size for list-decodable and list-recoverable codes with rates approaching capacity. Moreover, we show that list-decodable \emph{nonlinear} codes can strictly outperform list-decodable linear codes. More precisely, we show that there is a gap for a wide range of parameters, which grows fast with the alphabet size, between the size of the largest list-decodable nonlinear code and the size of the largest list-decodable linear codes. This is achieved by a novel connection between list-decoding and the notion of sparse hypergraphs in extremal combinatorics. We remark that such a gap is not known to exist in the problem of unique decoding. Lastly, we show that list-decodability or recoverability of codes implies in some sense good unique decodability.
PERF: Performant, Explicit Radiance Fields
Venue: arXiv cs.CV 2021
Year: 2021

We present a novel way of approaching image-based 3D reconstruction based on radiance fields. The problem of volumetric reconstruction is formulated as a non-linear least-squares problem and solved explicitly without the use of neural networks. This enables the use of solvers with a higher rate of convergence than what is typically used for neural networks, and fewer iterations are required until convergence. The volume is represented using a grid of voxels, with the scene surrounded by a hierarchy of environment maps. This makes it possible to get clean reconstructions of 360° scenes where the foreground and background is separated. A number of synthetic and real scenes from well known benchmark-suites are successfully reconstructed with quality on par with state-of-the-art methods, but at significantly reduced reconstruction times.
Interaction-Aware Sensitivity Analysis for Aerodynamic Optimization Results using Information Theory
Venue: arXiv cs.IT 2021
Year: 2021

An important issue during an engineering design process is to develop an understanding which design parameters have the most influence on the performance. Especially in the context of optimization approaches this knowledge is crucial in order to realize an efficient design process and achieve high-performing results. Information theory provides powerful tools to investigate these relationships because measures are model-free and thus also capture non-linear relationships, while requiring only minimal assumptions on the input data. We therefore propose to use recently introduced information-theoretic methods and estimation algorithms to find the most influential input parameters in optimization results. The proposed methods are in particular able to account for interactions between parameters, which are often neglected but may lead to redundant or synergistic contributions of multiple parameters. We demonstrate the application of these methods on optimization data from aerospace engineering, where we first identify the most relevant optimization parameters using a recently introduced information-theoretic feature-selection algorithm that accounts for interactions between parameters. Second, we use the novel partial information decomposition (PID) framework that allows to quantify redundant and synergistic contributions between selected parameters with respect to the optimization outcome to identify parameter interactions. We thus demonstrate the power of novel information-theoretic approaches in identifying relevant parameters in optimization runs and highlight how these methods avoid the selection of redundant parameters, while detecting interactions that result in synergistic contributions of multiple parameters.
Neural Multi-Quantile Forecasting for Optimal Inventory Management
Venue: arXiv cs.LG 2021
Year: 2021

In this work we propose the use of quantile regression and dilated recurrent neural networks with temporal scaling (MQ-DRNN-s) and apply it to the inventory management task. This model showed a better performance of up to 3.2\% over a statistical benchmark (the quantile autoregressive model with exogenous variables, QAR-X), being better than the MQ-DRNN without temporal scaling by 6\%. The above on a set of 10,000 time series of sales of El Globo over a 53-week horizon using rolling windows of 7-day ahead each week.
Formalizing Higher-Order Termination in Coq
Venue: arXiv cs.LO 2021
Year: 2021

We describe a formalization of higher-order rewriting theory and formally prove that an AFS is strongly normalizing if it can be interpreted in a well-founded domain. To do so, we use Coq, which is a proof assistant based on dependent type theory. Using this formalization, one can implement several termination techniques, like the interpretation method or dependency pairs, and prove their correctness. Those implementations can then be extracted to OCaml, which results in a verified termination checker.
Attacks on Wireless Coexistence: Exploiting Cross-Technology Performance Features for Inter-Chip Privilege Escalation
Venue: arXiv cs.CR 2021
Year: 2021

Modern mobile devices feature multiple wireless technologies, such as Bluetooth, Wi-Fi, and LTE. Each of them is implemented within a separate wireless chip, sometimes packaged as combo chips. However, these chips share components and resources, such as the same antenna or wireless spectrum. Wireless coexistence interfaces enable them to schedule packets without collisions despite shared resources, essential to maximizing networking performance. Today's hardwired coexistence interfaces hinder clear security boundaries and separation between chips and chip components. This paper shows practical coexistence attacks on Broadcom, Cypress, and Silicon Labs chips deployed in billions of devices. For example, we demonstrate that a Bluetooth chip can directly extract network passwords and manipulate traffic on a Wi-Fi chip. Coexistence attacks enable a novel type of lateral privilege escalation across chip boundaries. We responsibly disclosed the vulnerabilities to the vendors. Yet, only partial fixes were released for existing hardware since wireless chips would need to be redesigned from the ground up to prevent the presented attacks on coexistence.
Theory of and Experiments on Minimally Invasive Stability Preservation in Changing Two-Sided Matching Markets
Venue: arXiv cs.GT 2021
Year: 2021

Following up on purely theoretical work of Bredereck et al. [AAAI 2020], we contribute further theoretical insights into adapting stable two-sided matchings to change. Moreover, we perform extensive empirical studies hinting at numerous practically useful properties. Our theoretical extensions include the study of new problems (that is, incremental variants of Almost Stable Marriage and Hospital Residents), focusing on their (parameterized) computational complexity and the equivalence of various change types (thus simplifying algorithmic and complexity-theoretic studies for various natural change scenarios). Our experimental findings reveal, for instance, that allowing the new matching to be blocked by a few pairs significantly decreases the necessary differences between the old and the new stable matching.
TempoQR: Temporal Question Reasoning over Knowledge Graphs
Venue: arXiv cs.CL 2021
Year: 2021

Knowledge Graph Question Answering (KGQA) involves retrieving facts from a Knowledge Graph (KG) using natural language queries. A KG is a curated set of facts consisting of entities linked by relations. Certain facts include also temporal information forming a Temporal KG (TKG). Although many natural questions involve explicit or implicit time constraints, question answering (QA) over TKGs has been a relatively unexplored area. Existing solutions are mainly designed for simple temporal questions that can be answered directly by a single TKG fact. This paper puts forth a comprehensive embedding-based framework for answering complex questions over TKGs. Our method termed temporal question reasoning (TempoQR) exploits TKG embeddings to ground the question to the specific entities and time scope it refers to. It does so by augmenting the question embeddings with context, entity and time-aware information by employing three specialized modules. The first computes a textual representation of a given question, the second combines it with the entity embeddings for entities involved in the question, and the third generates question-specific time embeddings. Finally, a transformer-based encoder learns to fuse the generated temporal information with the question representation, which is used for answer predictions. Extensive experiments show that TempoQR improves accuracy by 25--45 percentage points on complex temporal questions over state-of-the-art approaches and it generalizes better to unseen question types.
A load balanced chemistry model with analytical Jacobian for faster reactive simulations in OpenFOAM
Venue: arXiv cs.CE 2021
Year: 2021

In this study, we introduce a novel open-source chemistry model for OpenFOAM to speed-up the reactive computational fluid dynamics (CFD) simulations using finite-rate chemistry. First, a dynamic load balancing model called DLBFoam is introduced to balance the chemistry load during runtime in parallel simulations. In addition, the solution of the cell-based chemistry problem is improved by utilizing an analytical Jacobian using an open-source library called pyJac and an efficient linear algebra library LAPACK. Combination of the aforementioned efforts yields a speed-up factor 200 for a high-fidelity large-eddy simulation spray combustion case compared to the standard OpenFOAM implementation. It is worth noting that the present implementation does not compromise the solution accuracy.