OpenRaiser/Intern-Atlas huggingface dataset:
Context-aware Visual Tracking with Joint Meta-updating
Venue: arXiv cs.CV 2022
Year: 2022

Visual object tracking acts as a pivotal component in various emerging video applications. Despite the numerous developments in visual tracking, existing deep trackers are still likely to fail when tracking against objects with dramatic variation. These deep trackers usually do not perform online update or update single sub-branch of the tracking model, for which they cannot adapt to the appearance variation of objects. Efficient updating methods are therefore crucial for tracking while previous meta-updater optimizes trackers directly over parameter space, which is prone to over-fit even collapse on longer sequences. To address these issues, we propose a context-aware tracking model to optimize the tracker over the representation space, which jointly meta-update both branches by exploiting information along the whole sequence, such that it can avoid the over-fitting problem. First, we note that the embedded features of the localization branch and the box-estimation branch, focusing on the local and global information of the target, are effective complements to each other. Based on this insight, we devise a context-aggregation module to fuse information in historical frames, followed by a context-aware module to learn affinity vectors for both branches of the tracker. Besides, we develop a dedicated meta-learning scheme, on account of fast and stable updating with limited training samples. The proposed tracking method achieves an EAO score of 0.514 on VOT2018 with the speed of 40FPS, demonstrating its capability of improving the accuracy and robustness of the underlying tracker with little speed drop.
Artificial Intelligence: Framework of driving triggers to past, present and future applications and influencers of industry sector adoption
Venue: arXiv cs.CY 2022
Year: 2022

To gain a sense of the development of Artificial Intelligence (AI), this research analyzes what has been done in the past, presently in the last decade and what is predicted for the next several decades. The paper will highlight the biggest changes in AI and give examples of how these technologies are applied in several key industry sectors along with influencers that can affect adoption speed. Lastly, the research examines the driving triggers such as cost, speed, accuracy, diversity/inclusion and interdisciplinary research/collaboration that propel AI into an essential transformative technology.
Bi-directional Loop Closure for Visual SLAM
Venue: arXiv cs.CV 2022
Year: 2022

A key functional block of visual navigation system for intelligent autonomous vehicles is Loop Closure detection and subsequent relocalisation. State-of-the-Art methods still approach the problem as uni-directional along the direction of the previous motion. As a result, most of the methods fail in the absence of a significantly similar overlap of perspectives. In this study, we propose an approach for bi-directional loop closure. This will, for the first time, provide us with the capability to relocalize to a location even when traveling in the opposite direction, thus significantly reducing long-term odometry drift in the absence of a direct loop. We present a technique to select training data from large datasets in order to make them usable for the bi-directional problem. The data is used to train and validate two different CNN architectures for loop closure detection and subsequent regression of 6-DOF camera pose between the views in an end-to-end manner. The outcome packs a considerable impact and aids significantly to real-world scenarios that do not offer direct loop closure opportunities. We provide a rigorous empirical comparison against other established approaches and evaluate our method on both outdoor and indoor data from the FinnForest dataset and PennCOSYVIO dataset.
SecureSense: Defending Adversarial Attack for Secure Device-Free Human Activity Recognition
Venue: arXiv cs.CR 2022
Year: 2022

Deep neural networks have empowered accurate device-free human activity recognition, which has wide applications. Deep models can extract robust features from various sensors and generalize well even in challenging situations such as data-insufficient cases. However, these systems could be vulnerable to input perturbations, i.e. adversarial attacks. We empirically demonstrate that both black-box Gaussian attacks and modern adversarial white-box attacks can render their accuracies to plummet. In this paper, we firstly point out that such phenomenon can bring severe safety hazards to device-free sensing systems, and then propose a novel learning framework, SecureSense, to defend common attacks. SecureSense aims to achieve consistent predictions regardless of whether there exists an attack on its input or not, alleviating the negative effect of distribution perturbation caused by adversarial attacks. Extensive experiments demonstrate that our proposed method can significantly enhance the model robustness of existing deep models, overcoming possible attacks. The results validate that our method works well on wireless human activity recognition and person identification systems. To the best of our knowledge, this is the first work to investigate adversarial attacks and further develop a novel defense framework for wireless human activity recognition in mobile computing research.
Introducing ECAPA-TDNN and Wav2Vec2.0 Embeddings to Stuttering Detection
Venue: arXiv cs.SD 2022
Year: 2022

The adoption of advanced deep learning (DL) architecture in stuttering detection (SD) tasks is challenging due to the limited size of the available datasets. To this end, this work introduces the application of speech embeddings extracted with pre-trained deep models trained on massive audio datasets for different tasks. In particular, we explore audio representations obtained using emphasized channel attention, propagation, and aggregation-time-delay neural network (ECAPA-TDNN) and Wav2Vec2.0 model trained on VoxCeleb and LibriSpeech datasets respectively. After extracting the embeddings, we benchmark with several traditional classifiers, such as a k-nearest neighbor, Gaussian naive Bayes, and neural network, for the stuttering detection tasks. In comparison to the standard SD system trained only on the limited SEP-28k dataset, we obtain a relative improvement of 16.74% in terms of overall accuracy over baseline. Finally, we have shown that combining two embeddings and concatenating multiple layers of Wav2Vec2.0 can further improve SD performance up to 1% and 2.64% respectively.
HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE
Venue: arXiv cs.CV 2022
Year: 2022

Studies on the automatic processing of 3D human pose data have flourished in the recent past. In this paper, we are interested in the generation of plausible and diverse future human poses following an observed 3D pose sequence. Current methods address this problem by injecting random variables from a single latent space into a deterministic motion prediction framework, which precludes the inherent multi-modality in human motion generation. In addition, previous works rarely explore the use of attention to select which frames are to be used to inform the generation process up to our knowledge. To overcome these limitations, we propose Hierarchical Transformer Dynamical Variational Autoencoder, HiT-DVAE, which implements auto-regressive generation with transformer-like attention mechanisms. HiT-DVAE simultaneously learns the evolution of data and latent space distribution with time correlated probabilistic dependencies, thus enabling the generative model to learn a more complex and time-varying latent space as well as diverse and realistic human motions. Furthermore, the auto-regressive generation brings more flexibility on observation and prediction, i.e. one can have any length of observation and predict arbitrary large sequences of poses with a single pre-trained model. We evaluate the proposed method on HumanEva-I and Human3.6M with various evaluation methods, and outperform the state-of-the-art methods on most of the metrics.
Looking for winning strategies in two-player games on Petri nets with partial observability
Venue: arXiv cs.FL 2022
Year: 2022

We define a game on 1-safe Petri nets, where a user plays against an environment in order to reach a goal on the system. The goal is expressed through an LTL-X formula, and represents a behaviour of the system that the user needs to guarantee. The user can try to reach his goal by controlling a subset of transitions, and by observing a subset of local states. Although we do not put any requirement on the local states observable by the user, we assume that he cannot be sure to observe them in the exact moment in which they become marked. For this reason, we define a notion of stability of the observation. We propose a method to determine whether the user has a winning strategy, i.e. if he can win every play by taking some decisions based on the information available for him.
MetaAID: A Flexible Framework for Developing Metaverse Applications via AI Technology and Human Editing
Venue: arXiv cs.CL 2022
Year: 2022

Achieving the expansion of domestic demand and the economic internal circulation requires balanced and coordinated support from multiple industries (domains) such as consumption, education, entertainment, engineering infrastructure, etc., which is indispensable for maintaining economic development. Metaverse applications may help with this task and can make many industries more interesting, more efficient, and provide a better user experience. The first challenge is that metaverse application development inevitably requires the support of various artificial intelligence (AI) technologies such as natural language processing (NLP), knowledge graph (KG), computer vision (CV), and machine learning (ML), etc. However, existing metaverse application development lacks a lightweight AI technology framework. This paper proposes a flexible metaverse AI technology framework metaAID that aims to support language and semantic technologies in the development of digital twins and virtual humans. The second challenge is that the development process of metaverse applications involves both technical development tasks and manual editing work, and often becomes a heavyweight multi-team collaboration project, not to mention the development of metaverse applications in multiple industries. Our framework summarizes common AI technologies and application development templates with common functional modules and interfaces. Based on this framework, we have designed 5 applications for 3 industries around the expansion of domestic demand and economic internal circulation. Experimental results show that our framework can support AI technologies when developing metaverse applications in different industries.
Semantic Similarity Metrics for Evaluating Source Code Summarization
Venue: arXiv cs.SE 2022
Year: 2022

Source code summarization involves creating brief descriptions of source code in natural language. These descriptions are a key component of software documentation such as JavaDocs. Automatic code summarization is a prized target of software engineering research, due to the high value summaries have to programmers and the simultaneously high cost of writing and maintaining documentation by hand. Current work is almost all based on machine models trained via big data input. Large datasets of examples of code and summaries of that code are used to train an e.g. encoder-decoder neural model. Then the output predictions of the model are evaluated against a set of reference summaries. The input is code not seen by the model, and the prediction is compared to a reference. The means by which a prediction is compared to a reference is essentially word overlap, calculated via a metric such as BLEU or ROUGE. The problem with using word overlap is that not all words in a sentence have the same importance, and many words have synonyms. The result is that calculated similarity may not match the perceived similarity by human readers. In this paper, we conduct an experiment to measure the degree to which various word overlap metrics correlate to human-rated similarity of predicted and reference summaries. We evaluate alternatives based on current work in semantic similarity metrics and propose recommendations for evaluation of source code summarization.
Contracts with Information Acquisition, via Scoring Rules
Venue: arXiv cs.GT 2022
Year: 2022

We consider a principal-agent problem where the agent may privately choose to acquire relevant information prior to taking a hidden action. This model generalizes two special cases: a classic moral hazard setting, and a more recently studied problem of incentivizing information acquisition (IA). We show that all of these problems can be reduced to the design of a proper scoring rule. Under a limited liability condition, we consider the special cases separately and then the general problem. We give novel results for the special case of IA, giving a closed form "pointed polyhedral cone" solution for the general multidimensional problem. We also describe a geometric, scoring-rules based solution to the case of the classic contracts problem. Finally, we give an efficient algorithm for the general problem of Contracts with Information Acquisition.
Automatic Text Summarization Methods: A Comprehensive Review
Venue: arXiv cs.CL 2022
Year: 2022

One of the most pressing issues that have arisen due to the rapid growth of the Internet is known as information overloading. Simplifying the relevant information in the form of a summary will assist many people because the material on any topic is plentiful on the Internet. Manually summarising massive amounts of text is quite challenging for humans. So, it has increased the need for more complex and powerful summarizers. Researchers have been trying to improve approaches for creating summaries since the 1950s, such that the machine-generated summary matches the human-created summary. This study provides a detailed state-of-the-art analysis of text summarization concepts such as summarization approaches, techniques used, standard datasets, evaluation metrics and future scopes for research. The most commonly accepted approaches are extractive and abstractive, studied in detail in this work. Evaluating the summary and increasing the development of reusable resources and infrastructure aids in comparing and replicating findings, adding competition to improve the outcomes. Different evaluation methods of generated summaries are also discussed in this study. Finally, at the end of this study, several challenges and research opportunities related to text summarization research are mentioned that may be useful for potential researchers working in this area.
A Data-Driven Framework for Identifying Investment Opportunities in Private Equity
Venue: arXiv cs.LG 2022
Year: 2022

The core activity of a Private Equity (PE) firm is to invest into companies in order to provide the investors with profit, usually within 4-7 years. To invest into a company or not is typically done manually by looking at various performance indicators of the company and then making a decision often based on instinct. This process is rather unmanageable given the large number of companies to potentially invest. Moreover, as more data about company performance indicators becomes available and the number of different indicators one may want to consider increases, manual crawling and assessment of investment opportunities becomes inefficient and ultimately impossible. To address these issues, this paper proposes a framework for automated data-driven screening of investment opportunities and thus the recommendation of businesses to invest in. The framework draws on data from several sources to assess the financial and managerial position of a company, and then uses an explainable artificial intelligence (XAI) engine to suggest investment recommendations. The robustness of the model is validated using different AI algorithms, class imbalance-handling methods, and features extracted from the available data sources.
Multitasking Scheduling with Shared Processing
Venue: arXiv cs.DS 2022
Year: 2022

Recently, the problem of multitasking scheduling has attracted a lot of attention in the service industries where workers frequently perform multiple tasks by switching from one task to another. Hall, Leung and Li (Discrete Applied Mathematics 2016) proposed a shared processing multitasking scheduling model which allows a team to continue to work on the primary tasks while processing the routinely scheduled activities as they occur. The processing sharing is achieved by allocating a fraction of the processing capacity to routine jobs and the remaining fraction, which we denote as sharing ratio, to the primary jobs. In this paper, we generalize this model to parallel machines and allow the fraction of the processing capacity assigned to routine jobs to vary from one to another. The objectives are minimizing makespan and minimizing the total completion time. We show that for both objectives, there is no polynomial time approximation algorithm unless P=NP if the sharing ratios are arbitrary for all machines. Then we consider the problems where the sharing ratios on some machines have a constant lower bound. For each objective, we analyze the performance of the classical scheduling algorithms and their variations and then develop a polynomial time approximation scheme when the number of machines is a constant.
Internet of Things: System Reference Architecture
Venue: arXiv cs.DC 2022
Year: 2022

This document describes an IoT system reference architecture. It is a general system architecture in the sense that it embodies key system components, functions, and flows that are commonly encountered in IoT systems. Much of the existing literature on IoT system architecture covers either specific commercial solutions or abstract architecture descriptions that are conceptual but not readily reducible to practice. This paper describes a general, vendor-neutral architecture of the IoT system derived by abstracting the common requirements and features of a variety of system specifications and implementations. It covers architectural and design principles for constructing the core IoT system overlay, specifically functions and components involved in acquiring data, real-time processing, storage and delivery to applications and services, such as visualization and AI. The appendix outlines the principles of IoT information modeling and data and metadata handling. As a reference architecture, this document is meant to serve as a template and a starting blueprint for constructing specific IoT system solutions. The intent is to maintain this material as a living document and evolve it over time into a broader technical community consensus by incorporating and acknowledging the accumulated feedback and contributions. Direct comments and suggestions are invited now. More structured mechanisms of managing crowdsourcing and community collaboration are planned for implementation if warranted.
Dealing with complex routing requirements using an MCDM based approach
Venue: arXiv cs.NI 2022
Year: 2022

The last decade has witnessed an ever-growing user demand for a better QoS (Quality Of Service) and the fast growth of connected devices still put high pressure on the legacy network infrastructures. To improve network performances, better manage the resources and have a greater control over traffic transmission, intelligent routing procedures are increasingly demanded. Modern applications in the dynamic context of new emerging networks have their own routing requirements, in terms of set of metrics to consider, their importance and thresholds to respect. The objective of this work is to design an approach based on MCDM (Multi-Criteria Decision Making) to decide complex routing problems when assuming threshold constraints on metrics. We give the mathematical framework to capture such requirements and to decide the routing. A case study is presented to advocate the benefit of using our approach.
Domain-Aware Contrastive Knowledge Transfer for Multi-domain Imbalanced Data
Venue: arXiv cs.LG 2022
Year: 2022

In many real-world machine learning applications, samples belong to a set of domains e.g., for product reviews each review belongs to a product category. In this paper, we study multi-domain imbalanced learning (MIL), the scenario that there is imbalance not only in classes but also in domains. In the MIL setting, different domains exhibit different patterns and there is a varying degree of similarity and divergence among domains posing opportunities and challenges for transfer learning especially when faced with limited or insufficient training data. We propose a novel domain-aware contrastive knowledge transfer method called DCMI to (1) identify the shared domain knowledge to encourage positive transfer among similar domains (in particular from head domains to tail domains); (2) isolate the domain-specific knowledge to minimize the negative transfer from dissimilar domains. We evaluated the performance of DCMI on three different datasets showing significant improvements in different MIL scenarios.
Text Spotting Transformers
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present TExt Spotting TRansformers (TESTR), a generic end-to-end text spotting framework using Transformers for text detection and recognition in the wild. TESTR builds upon a single encoder and dual decoders for the joint text-box control point regression and character recognition. Other than most existing literature, our method is free from Region-of-Interest operations and heuristics-driven post-processing procedures; TESTR is particularly effective when dealing with curved text-boxes where special cares are needed for the adaptation of the traditional bounding-box representations. We show our canonical representation of control points suitable for text instances in both Bezier curve and polygon annotations. In addition, we design a bounding-box guided polygon detection (box-to-polygon) process. Experiments on curved and arbitrarily shaped datasets demonstrate state-of-the-art performances of the proposed TESTR algorithm.
Attention Distraction: Watermark Removal Through Continual Learning with Selective Forgetting
Venue: arXiv cs.CV 2022
Year: 2022

Fine-tuning attacks are effective in removing the embedded watermarks in deep learning models. However, when the source data is unavailable, it is challenging to just erase the watermark without jeopardizing the model performance. In this context, we introduce Attention Distraction (AD), a novel source data-free watermark removal attack, to make the model selectively forget the embedded watermarks by customizing continual learning. In particular, AD first anchors the model's attention on the main task using some unlabeled data. Then, through continual learning, a small number of \textit{lures} (randomly selected natural images) that are assigned a new label distract the model's attention away from the watermarks. Experimental results from different datasets and networks corroborate that AD can thoroughly remove the watermark with a small resource budget without compromising the model's performance on the main task, which outperforms the state-of-the-art works.