OpenRaiser/Intern-Atlas huggingface dataset:
GRASP EARTH: Intuitive Software for Discovering Changes on the Planet
Venue: arXiv cs.CV 2022
Year: 2022

Detecting changes on the Earth, such as urban development, deforestation, or natural disaster, is one of the research fields that is attracting a great deal of attention. One promising tool to solve these problems is satellite imagery. However, satellite images require huge amount of storage, therefore users are required to set Area of Interests first, which was not suitable for detecting potential areas for disaster or development. To tackle with this problem, we develop the novel tool, namely GRASP EARTH, which is the simple change detection application based on Google Earth Engine. GRASP EARTH allows us to handle satellite imagery easily and it has used for disaster monitoring and urban development monitoring.
Aggregated Pyramid Vision Transformer: Split-transform-merge Strategy for Image Recognition without Convolutions
Venue: arXiv cs.CV 2022
Year: 2022

With the achievements of Transformer in the field of natural language processing, the encoder-decoder and the attention mechanism in Transformer have been applied to computer vision. Recently, in multiple tasks of computer vision (image classification, object detection, semantic segmentation, etc.), state-of-the-art convolutional neural networks have introduced some concepts of Transformer. This proves that Transformer has a good prospect in the field of image recognition. After Vision Transformer was proposed, more and more works began to use self-attention to completely replace the convolutional layer. This work is based on Vision Transformer, combined with the pyramid architecture, using Split-transform-merge to propose the group encoder and name the network architecture Aggregated Pyramid Vision Transformer (APVT). We perform image classification tasks on the CIFAR-10 dataset and object detection tasks on the COCO 2017 dataset. Compared with other network architectures that use Transformer as the backbone, APVT has excellent results while reducing the computational cost. We hope this improved strategy can provide a reference for future Transformer research in computer vision.
The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking
Venue: arXiv cs.CL 2022
Year: 2022

Chinese Spell Checking (CSC) aims to detect and correct Chinese spelling errors, which are mainly caused by the phonological or visual similarity. Recently, pre-trained language models (PLMs) promote the progress of CSC task. However, there exists a gap between the learned knowledge of PLMs and the goal of CSC task. PLMs focus on the semantics in text and tend to correct the erroneous characters to semantically proper or commonly used ones, but these aren't the ground-truth corrections. To address this issue, we propose an Error-driven COntrastive Probability Optimization (ECOPO) framework for CSC task. ECOPO refines the knowledge representations of PLMs, and guides the model to avoid predicting these common characters through an error-driven way. Particularly, ECOPO is model-agnostic and it can be combined with existing CSC methods to achieve better performance. Extensive experiments and detailed analyses on SIGHAN datasets demonstrate that ECOPO is simple yet effective.
On the Configuration of More and Less Expressive Logic Programs
Venue: arXiv cs.AI 2022
Year: 2022

The decoupling between the representation of a certain problem, i.e., its knowledge model, and the reasoning side is one of main strong points of model-based Artificial Intelligence (AI). This allows, e.g. to focus on improving the reasoning side by having advantages on the whole solving process. Further, it is also well-known that many solvers are very sensitive to even syntactic changes in the input. In this paper, we focus on improving the reasoning side by taking advantages of such sensitivity. We consider two well-known model-based AI methodologies, SAT and ASP, define a number of syntactic features that may characterise their inputs, and use automated configuration tools to reformulate the input formula or program. Results of a wide experimental analysis involving SAT and ASP domains, taken from respective competitions, show the different advantages that can be obtained by using input reformulation and configuration. Under consideration in Theory and Practice of Logic Programming (TPLP).
ReZone: Disarming TrustZone with TEE Privilege Reduction
Venue: arXiv cs.CR 2022
Year: 2022

In TrustZone-assisted TEEs, the trusted OS has unrestricted access to both secure and normal world memory. Unfortunately, this architectural limitation has opened an aisle of exploration for attackers, which have demonstrated how to leverage a chain of exploits to hijack the trusted OS and gain full control of the system, targeting (i) the rich execution environment (REE), (ii) all trusted applications (TAs), and (iii) the secure monitor. In this paper, we propose ReZone. The main novelty behind ReZone design relies on leveraging TrustZone-agnostic hardware primitives available on commercially off-the-shelf (COTS) platforms to restrict the privileges of the trusted OS. With ReZone, a monolithic TEE is restructured and partitioned into multiple sandboxed domains named zones, which have only access to private resources. We have fully implemented ReZone for the i.MX 8MQuad EVK and integrated it with Android OS and OP-TEE. We extensively evaluated ReZone using microbenchmarks and real-world applications. ReZone can sustain popular applications like DRM-protected video encoding with acceptable performance overheads. We have surveyed 80 CVE vulnerability reports and estimate that ReZone could mitigate 86.84% of them.
Image-based material analysis of ancient historical documents
Venue: arXiv cs.CV 2022
Year: 2022

Researchers continually perform corroborative tests to classify ancient historical documents based on the physical materials of their writing surfaces. However, these tests, often performed on-site, requires actual access to the manuscript objects. The procedures involve a considerable amount of time and cost, and can damage the manuscripts. Developing a technique to classify such documents using only digital images can be very useful and efficient. In order to tackle this problem, this study uses images of a famous historical collection, the Dead Sea Scrolls, to propose a novel method to classify the materials of the manuscripts. The proposed classifier uses the two-dimensional Fourier Transform to identify patterns within the manuscript surfaces. Combining a binary classification system employing the transform with a majority voting process is shown to be effective for this classification task. This pilot study shows a successful classification percentage of up to 97% for a confined amount of manuscripts produced from either parchment or papyrus material. Feature vectors based on Fourier-space grid representation outperformed a concentric Fourier-space format.
Discontinuous Constituency and BERT: A Case Study of Dutch
Venue: arXiv cs.CL 2022
Year: 2022

In this paper, we set out to quantify the syntactic capacity of BERT in the evaluation regime of non-context free patterns, as occurring in Dutch. We devise a test suite based on a mildly context-sensitive formalism, from which we derive grammars that capture the linguistic phenomena of control verb nesting and verb raising. The grammars, paired with a small lexicon, provide us with a large collection of naturalistic utterances, annotated with verb-subject pairings, that serve as the evaluation test bed for an attention-based span selection probe. Our results, backed by extensive analysis, suggest that the models investigated fail in the implicit acquisition of the dependencies examined.
Algebraic Representations of Unique Bipartite Perfect Matching
Venue: arXiv cs.DM 2022
Year: 2022

We obtain complete characterizations of the Unique Bipartite Perfect Matching function, and of its Boolean dual, using multilinear polynomials over the reals. Building on previous results, we show that, surprisingly, the dual description is sparse and has low $\ell_1$-norm -- only exponential in $Θ(n \log n)$, and this result extends even to other families of matching-related functions. Our approach relies on the Möbius numbers in the matching-covered lattice, and a key ingredient in our proof is Möbius' inversion formula. These polynomial representations yield complexity-theoretic results. For instance, we show that unique bipartite matching is evasive for classical decision trees, and nearly evasive even for generalized query models. We also obtain a tight $Θ(n \log n)$ bound on the log-rank of the associated two-party communication task.
Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Environment perception in autonomous driving vehicles often heavily relies on deep neural networks (DNNs), which are subject to domain shifts, leading to a significantly decreased performance during DNN deployment. Usually, this problem is addressed by unsupervised domain adaptation (UDA) approaches trained either simultaneously on source and target domain datasets or even source-free only on target data in an offline fashion. In this work, we further expand a source-free UDA approach to a continual and therefore online-capable UDA on a single-image basis for semantic segmentation. Accordingly, our method only requires the pre-trained model from the supplier (trained in the source domain) and the current (unlabeled target domain) camera image. Our method Continual BatchNorm Adaptation (CBNA) modifies the source domain statistics in the batch normalization layers, using target domain images in an unsupervised fashion, which yields consistent performance improvements during inference. Thereby, in contrast to existing works, our approach can be applied to improve a DNN continuously on a single-image basis during deployment without access to source data, without algorithmic delay, and nearly without computational overhead. We show the consistent effectiveness of our method across a wide variety of source/target domain settings for semantic segmentation. Code is available at https://github.com/ifnspaml/CBNA.
SFCaaS: Service Function Chains as a Service in NFV Environments
Venue: arXiv cs.NI 2022
Year: 2022

With the emergence of network softwarization trend, traditional networking services offered by Internet providers are expected to evolve by fully leveraging new recent technologies like network function virtualization and software defined networking. In this paper, we investigate offering Service Function Chains as a Service (SFCaaS) in NFV Environments. We first describe the potential business model to offer such a service. We then conduct a detailed study of the costs of virtual machine instances offered by Amazon EC2 with respect to the location, instance size, and performance in order to guide service chain provisioning and resource allocation. Afterwards, we address the resource allocation problem for service chain functions from the SFC provider's perspective while leveraging the performed cost study. We hence formulate the problem as an Integer Linear Program (ILP) aiming at reducing the SFC provider's operational costs of virtual machine instances and links as well as the synchronization costs among the instances. We also propose a new heuristic algorithm to solve the mapping problem with the same aforementioned goals taking into account the conducted study of the costs of Amazon EC2 instances. We show through extensive simulations that the proposed heuristic significantly reduce operational costs compared to a Baseline algorithm inspired by the existing literature.
Self-Supervised Scene Flow Estimation with 4-D Automotive Radar
Venue: arXiv cs.CV 2022
Year: 2022

Scene flow allows autonomous vehicles to reason about the arbitrary motion of multiple independent objects which is the key to long-term mobile autonomy. While estimating the scene flow from LiDAR has progressed recently, it remains largely unknown how to estimate the scene flow from a 4-D radar - an increasingly popular automotive sensor for its robustness against adverse weather and lighting conditions. Compared with the LiDAR point clouds, radar data are drastically sparser, noisier and in much lower resolution. Annotated datasets for radar scene flow are also in absence and costly to acquire in the real world. These factors jointly pose the radar scene flow estimation as a challenging problem. This work aims to address the above challenges and estimate scene flow from 4-D radar point clouds by leveraging self-supervised learning. A robust scene flow estimation architecture and three novel losses are bespoken designed to cope with intractable radar data. Real-world experimental results validate that our method is able to robustly estimate the radar scene flow in the wild and effectively supports the downstream task of motion segmentation.
Controlling the Focus of Pretrained Language Generation Models
Venue: arXiv cs.AI 2022
Year: 2022

The finetuning of pretrained transformer-based language generation models are typically conducted in an end-to-end manner, where the model learns to attend to relevant parts of the input by itself. However, there does not exist a mechanism to directly control the model's focus. This work aims to develop a control mechanism by which a user can select spans of context as "highlights" for the model to focus on, and generate relevant output. To achieve this goal, we augment a pretrained model with trainable "focus vectors" that are directly applied to the model's embeddings, while the model itself is kept fixed. These vectors, trained on automatic annotations derived from attribution methods, act as indicators for context importance. We test our approach on two core generation tasks: dialogue response generation and abstractive summarization. We also collect evaluation data where the highlight-generation pairs are annotated by humans. Our experiments show that the trained focus vectors are effective in steering the model to generate outputs that are relevant to user-selected highlights.
EnDSUM: Entropy and Diversity based Disaster Tweet Summarization
Venue: arXiv cs.SI 2022
Year: 2022

The huge amount of information shared in Twitter during disaster events are utilized by government agencies and humanitarian organizations to ensure quick crisis response and provide situational updates. However, the huge number of tweets posted makes manual identification of the relevant tweets impossible. To address the information overload, there is a need to automatically generate summary of all the tweets which can highlight the important aspects of the disaster. In this paper, we propose an entropy and diversity based summarizer, termed as EnDSUM, specifically for disaster tweet summarization. Our comprehensive analysis on 6 datasets indicates the effectiveness of EnDSUM and additionally, highlights the scope of improvement of EnDSUM.
VAE-iForest: Auto-encoding Reconstruction and Isolation-based Anomalies Detecting Fallen Objects on Road Surface
Venue: arXiv cs.CV 2022
Year: 2022

In road monitoring, it is an important issue to detect changes in the road surface at an early stage to prevent damage to third parties. The target of the falling object may be a fallen tree due to the external force of a flood or an earthquake, and falling rocks from a slope. Generative deep learning is possible to flexibly detect anomalies of the falling objects on the road surface. We prototype a method that combines auto-encoding reconstruction and isolation-based anomaly detector in application for road surface monitoring. Actually, we apply our method to a set of test images that fallen objects is located on the raw inputs added with fallen stone and plywood, and that snow is covered on the winter road. Finally we mention the future works for practical purpose application.
Theoretical Foundation of Colored Petri Net through an Analysis of their Markings as Multi-classification
Venue: arXiv cs.LO 2022
Year: 2022

Barwise and Seligman stated the first principle of information flow: "Information flow results from regularities in the distributed system." They represent a distributed system in terms of a classification consisting of a set of objects or tokens to be classified, a set of types used to classify tokens, and a binary relation between tokens and types that tells one which tokens are classified as being of which types. We aim to further this investigation and proceed with a dynamic or evolving system instead of a static system. We claim that a classification is a snapshot of a distributed system at a given moment or context. We then aim to answer the question posed by an evolving context. As the context or configuration changes, how to regularities evolve. This paper is a continuation of an investigation we started in \cite{esterlin}, where we initiated how to capture a dynamism of information flow with a Kripke structure. Here we develop the same procedure with colored Petri net(CPN). We first extend the classification concept to multiclassification by replacing its binary relation between tokens and types with a multi relation: a function from tok(A) x typ(A) to N, the set of natural numbers. The multiclassification will unfold into binary classification in order to compute its theory. It turns out that markings of a CPN are multiclassification; Amalgamating the theories of those classifications obtained as markings of CPN results in a CPN's knowledge base.
Audio Self-supervised Learning: A Survey
Venue: arXiv cs.SD 2022
Year: 2022

Inspired by the humans' cognitive ability to generalise knowledge and skills, Self-Supervised Learning (SSL) targets at discovering general representations from large-scale data without requiring human annotations, which is an expensive and time consuming task. Its success in the fields of computer vision and natural language processing have prompted its recent adoption into the field of audio and speech processing. Comprehensive reviews summarising the knowledge in audio SSL are currently missing. To fill this gap, in the present work, we provide an overview of the SSL methods used for audio and speech processing applications. Herein, we also summarise the empirical works that exploit the audio modality in multi-modal SSL frameworks, and the existing suitable benchmarks to evaluate the power of SSL in the computer audition domain. Finally, we discuss some open problems and point out the future directions on the development of audio SSL.
On the application of generative adversarial networks for nonlinear modal analysis
Venue: arXiv cs.LG 2022
Year: 2022

Linear modal analysis is a useful and effective tool for the design and analysis of structures. However, a comprehensive basis for nonlinear modal analysis remains to be developed. In the current work, a machine learning scheme is proposed with a view to performing nonlinear modal analysis. The scheme is focussed on defining a one-to-one mapping from a latent `modal' space to the natural coordinate space, whilst also imposing orthogonality of the mode shapes. The mapping is achieved via the use of the recently-developed cycle-consistent generative adversarial network (cycle-GAN) and an assembly of neural networks targeted on maintaining the desired orthogonality. The method is tested on simulated data from structures with cubic nonlinearities and different numbers of degrees of freedom, and also on data from an experimental three-degree-of-freedom set-up with a column-bumper nonlinearity. The results reveal the method's efficiency in separating the `modes'. The method also provides a nonlinear superposition function, which in most cases has very good accuracy.
Low-Degree Multicalibration
Venue: arXiv cs.LG 2022
Year: 2022

Introduced as a notion of algorithmic fairness, multicalibration has proved to be a powerful and versatile concept with implications far beyond its original intent. This stringent notion -- that predictions be well-calibrated across a rich class of intersecting subpopulations -- provides its strong guarantees at a cost: the computational and sample complexity of learning multicalibrated predictors are high, and grow exponentially with the number of class labels. In contrast, the relaxed notion of multiaccuracy can be achieved more efficiently, yet many of the most desirable properties of multicalibration cannot be guaranteed assuming multiaccuracy alone. This tension raises a key question: Can we learn predictors with multicalibration-style guarantees at a cost commensurate with multiaccuracy? In this work, we define and initiate the study of Low-Degree Multicalibration. Low-Degree Multicalibration defines a hierarchy of increasingly-powerful multi-group fairness notions that spans multiaccuracy and the original formulation of multicalibration at the extremes. Our main technical contribution demonstrates that key properties of multicalibration, related to fairness and accuracy, actually manifest as low-degree properties. Importantly, we show that low-degree multicalibration can be significantly more efficient than full multicalibration. In the multi-class setting, the sample complexity to achieve low-degree multicalibration improves exponentially (in the number of classes) over full multicalibration. Our work presents compelling evidence that low-degree multicalibration represents a sweet spot, pairing computational and sample efficiency with strong fairness and accuracy guarantees.
Far-UVC Disinfection with Robotic Mobile Manipulator
Venue: arXiv cs.RO 2022
Year: 2022

The COVID-19 pandemic has demonstrated the need for a more effective and efficient disinfection approach to combat infectious diseases. Ultraviolet germicidal irradiation (UVGI) is a proven mean for disinfection and sterilization and has been integrated into handheld devices and autonomous mobile robots. Existing UVGI robots which are commonly equipped with uncovered lamps that emit intense ultraviolet radiation suffer from: inability to be used in human presence, shadowing of objects, and long disinfection time. These robots also have a high operational cost. This paper introduces a cost-effective germicidal system that utilizes UVGI to disinfect pathogens, such as viruses, bacteria, and fungi, on high contact surfaces (e.g. doors and tables). This system is composed of a team of 5-DOF mobile manipulators with end-effectors that are equipped with far-UVC excimer lamps. The design of the system is discussed with emphasis on path planning, coverage planning, and scene understanding. Evaluations of the UVGI system using simulations and irradiance models are also included.