OpenRaiser/Intern-Atlas huggingface dataset:
ASAP: Accurate semantic segmentation for real time performance
Venue: arXiv cs.CV 2022
Year: 2022

Feature fusion modules from encoder and self-attention module have been adopted in semantic segmentation. However, the computation of these modules is costly and has operational limitations in real-time environments. In addition, segmentation performance is limited in autonomous driving environments with a lot of contextual information perpendicular to the road surface, such as people, buildings, and general objects. In this paper, we propose an efficient feature fusion method, Feature Fusion with Different Norms (FFDN) that utilizes rich global context of multi-level scale and vertical pooling module before self-attention that preserves most contextual information while reducing the complexity of global context encoding in the vertical direction. By doing this, we could handle the properties of representation in global space and reduce additional computational cost. In addition, we analyze low performance in challenging cases including small and vertically featured objects. We achieve the mean Interaction of-union(mIoU) of 73.1 and the Frame Per Second(FPS) of 191, which are comparable results with state-of-the-arts on Cityscapes test datasets.
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Gate functions in recurrent models, such as an LSTM and GRU, play a central role in learning various time scales in modeling time series data by using a bounded activation function. However, it is difficult to train gates to capture extremely long time scales due to gradient vanishing of the bounded function for large inputs, which is known as the saturation problem. We closely analyze the relation between saturation of the gate function and efficiency of the training. We prove that the gradient vanishing of the gate function can be mitigated by accelerating the convergence of the saturating function, i.e., making the output of the function converge to 0 or 1 faster. Based on the analysis results, we propose a gate function called fast gate that has a doubly exponential convergence rate with respect to inputs by simple function composition. We empirically show that our method outperforms previous methods in accuracy and computational efficiency on benchmark tasks involving extremely long time scales.
Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies
Venue: arXiv cs.HC 2022
Year: 2022

As virtual reality (VR) technologies have improved in the past decade, more research has investigated how they could support more effective communication in various contexts to improve collaboration and social connectedness. However, there was no literature to summarize the uniqueness VR provided and put forward guidance for designing social VR applications for better communication. To understand how VR has been designed and used to facilitate communication in different contexts, we conducted a systematic review of the studies investigating communication in social VR in the past ten years by following the PRISMA guidelines. We highlight current practices and challenges and identify research opportunities to improve the design of social VR to better support communication and make social VR more accessible.
Learning Deep Nets for Gravitational Dynamics with Unknown Disturbance through Physical Knowledge Distillation: Initial Feasibility Study
Venue: arXiv cs.RO 2022
Year: 2022

Learning high-performance deep neural networks for dynamic modeling of high Degree-Of-Freedom (DOF) robots remains challenging due to the sampling complexity. Typical unknown system disturbance caused by unmodeled dynamics (such as internal compliance, cables) further exacerbates the problem. In this paper, a novel framework characterized by both high data efficiency and disturbance-adapting capability is proposed to address the problem of modeling gravitational dynamics using deep nets in feedforward gravity compensation control for high-DOF master manipulators with unknown disturbance. In particular, Feedforward Deep Neural Networks (FDNNs) are learned from both prior knowledge of an existing analytical model and observation of the robot system by Knowledge Distillation (KD). Through extensive experiments in high-DOF master manipulators with significant disturbance, we show that our method surpasses a standard Learning-from-Scratch (LfS) approach in terms of data efficiency and disturbance adaptation. Our initial feasibility study has demonstrated the potential of outperforming the analytical teacher model as the training data increases.
A Compact Model of Interface-Type Memristors Linking Physical and Device Properties
Venue: arXiv cs.ET 2022
Year: 2022

Memristors are an electronic device whose resistance depends on the voltage history that has been applied to its two terminals. Despite its clear advantage as a computational element, a suitable transport model is lacking for the special class of interface-based memristors. Here, we adapt the widely-used Yakopcic compact model by including transport equations relevant to interface-type memristors. This model is able to reproduce the qualitative behaviour measured upon Nb-doped SrTiO$_3$ memristive devices. Our analysis demonstrates a direct correlation between the devices' characteristic parameters and those of our model. The model can clearly identify the charge transport mechanism in different resistive states thus facilitating evaluation of the relevant parameters pertaining to resistive switching in interface-based memristors. One clear application of our study is its ability to inform the design and fabrication of related memristive devices.
CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Training deep learning models on medical datasets that perform well for all classes is a challenging task. It is often the case that a suboptimal performance is obtained on some classes due to the natural class imbalance issue that comes with medical data. An effective way to tackle this problem is by using targeted active learning, where we iteratively add data points to the training data that belong to the rare classes. However, existing active learning methods are ineffective in targeting rare classes in medical datasets. In this work, we propose Clinical (targeted aCtive Learning for ImbalaNced medICal imAge cLassification) a framework that uses submodular mutual information functions as acquisition functions to mine critical data points from rare classes. We apply our framework to a wide-array of medical imaging datasets on a variety of real-world class imbalance scenarios - namely, binary imbalance and long-tail imbalance. We show that Clinical outperforms the state-of-the-art active learning methods by acquiring a diverse set of data points that belong to the rare classes.
Scheduling with Many Shared Resources
Venue: arXiv cs.DS 2022
Year: 2022

Consider the many shared resource scheduling problem where jobs have to be scheduled on identical parallel machines with the goal of minimizing the makespan. However, each job needs exactly one additional shared resource in order to be executed and hence prevents the execution of jobs that need the same resource while being processed. Previously a $(2m/(m+1))$-approximation was the best known result for this problem. Furthermore, a $6/5$-approximation for the case with only two machines was known as well as a PTAS for the case with a constant number of machines. We present a simple and fast 5/3-approximation and a much more involved but still reasonable 1.5-approximation. Furthermore, we provide a PTAS for the case with only a constant number of machines, which is arguably simpler and faster than the previously known one, as well as a PTAS with resource augmentation for the general case. The approximation schemes make use of the N-fold integer programming machinery, which has found more and more applications in the field of scheduling recently. It is plausible that the latter results can be improved and extended to more general cases. Lastly, we give a $5/4 - \varepsilon$ inapproximability result for the natural problem extension where each job may need up to a constant number (in particular $3$) of different resources.
Concise and interpretable multi-label rule sets
Venue: arXiv cs.LG 2022
Year: 2022

Multi-label classification is becoming increasingly ubiquitous, but not much attention has been paid to interpretability. In this paper, we develop a multi-label classifier that can be represented as a concise set of simple "if-then" rules, and thus, it offers better interpretability compared to black-box models. Notably, our method is able to find a small set of relevant patterns that lead to accurate multi-label classification, while existing rule-based classifiers are myopic and wasteful in searching rules,requiring a large number of rules to achieve high accuracy. In particular, we formulate the problem of choosing multi-label rules to maximize a target function, which considers not only discrimination ability with respect to labels, but also diversity. Accounting for diversity helps to avoid redundancy, and thus, to control the number of rules in the solution set. To tackle the said maximization problem we propose a 2-approximation algorithm, which relies on a novel technique to sample high-quality rules. In addition to our theoretical analysis, we provide a thorough experimental evaluation, which indicates that our approach offers a trade-off between predictive performance and interpretability that is unmatched in previous work.
Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.
SicHash -- Small Irregular Cuckoo Tables for Perfect Hashing
Venue: arXiv cs.DS 2022
Year: 2022

A Perfect Hash Function (PHF) is a hash function that has no collisions on a given input set. PHFs can be used for space efficient storage of data in an array, or for determining a compact representative of each object in the set. In this paper, we present the PHF construction algorithm SicHash - Small Irregular Cuckoo Tables for Perfect Hashing. At its core, SicHash uses a known technique: It places objects in a cuckoo hash table and then stores the final hash function choice of each object in a retrieval data structure. We combine the idea with irregular cuckoo hashing, where each object has a different number of hash functions. Additionally, we use many small tables that we overload beyond their asymptotic maximum load factor. The most space efficient competitors often use brute force methods to determine the PHFs. SicHash provides a more direct construction algorithm that only rarely needs to recompute parts. Our implementation improves the state of the art in terms of space usage versus construction time for a wide range of configurations. At the same time, it provides very fast queries.
Cross-Geography Generalization of Machine Learning Methods for Classification of Flooded Regions in Aerial Images
Venue: arXiv cs.CV 2022
Year: 2022

Identification of regions affected by floods is a crucial piece of information required for better planning and management of post-disaster relief and rescue efforts. Traditionally, remote sensing images are analysed to identify the extent of damage caused by flooding. The data acquired from sensors onboard earth observation satellites are analyzed to detect the flooded regions, which can be affected by low spatial and temporal resolution. However, in recent years, the images acquired from Unmanned Aerial Vehicles (UAVs) have also been utilized to assess post-disaster damage. Indeed, a UAV based platform can be rapidly deployed with a customized flight plan and minimum dependence on the ground infrastructure. This work proposes two approaches for identifying flooded regions in UAV aerial images. The first approach utilizes texture-based unsupervised segmentation to detect flooded areas, while the second uses an artificial neural network on the texture features to classify images as flooded and non-flooded. Unlike the existing works where the models are trained and tested on images of the same geographical regions, this work studies the performance of the proposed model in identifying flooded regions across geographical regions. An F1-score of 0.89 is obtained using the proposed segmentation-based approach which is higher than existing classifiers. The robustness of the proposed approach demonstrates that it can be utilized to identify flooded regions of any region with minimum or no user intervention.
Self-improving Multiplane-to-layer Images for Novel View Synthesis
Venue: arXiv cs.CV 2022
Year: 2022

We present a new method for lightweight novel-view synthesis that generalizes to an arbitrary forward-facing scene. Recent approaches are computationally expensive, require per-scene optimization, or produce a memory-expensive representation. We start by representing the scene with a set of fronto-parallel semitransparent planes and afterward convert them to deformable layers in an end-to-end manner. Additionally, we employ a feed-forward refinement procedure that corrects the estimated representation by aggregating information from input views. Our method does not require fine-tuning when a new scene is processed and can handle an arbitrary number of views without restrictions. Experimental results show that our approach surpasses recent models in terms of common metrics and human evaluation, with the noticeable advantage in inference speed and compactness of the inferred layered geometry, see https://samsunglabs.github.io/MLI
Opening Access to Visual Exploration of Audiovisual Digital Biomarkers: an OpenDBM Analytics Tool
Venue: arXiv cs.HC 2022
Year: 2022

Digital biomarkers (DBMs) are a growing field and increasingly tested in the therapeutic areas of psychiatric and neurodegenerative disorders. Meanwhile, isolated silos of knowledge of audiovisual DBMs use in industry, academia, and clinics hinder their widespread adoption in clinical research. How can we help these non-technical domain experts to explore audiovisual digital biomarkers? The use of open source software in biomedical research to extract patient behavior changes is growing and inspiring a shift toward accessibility to address this problem. OpenDBM integrates several popular audio and visual open source behavior extraction toolkits. We present a visual analysis tool as an extension of the growing open source software, OpenDBM, to promote the adoption of audiovisual DBMs in basic and applied research. Our tool illustrates patterns in behavioral data while supporting interactive visual analysis of any subset of derived or raw DBM variables extracted through OpenDBM.
2nd Place Solution to Google Landmark Retrieval 2020
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents the 2nd place solution to the Google Landmark Retrieval Competition 2020. We propose a training method of global feature model for landmark retrieval without post-processing, such as local feature and spatial verification. There are two parts in our retrieval method in this competition. This training scheme mainly includes training by increasing margin value of arcmargin loss and increasing image resolution step by step. Models are trained by PaddlePaddle framework and Pytorch framework, and then converted to tensorflow 2.2. Using this method, we got a public score of 0.40176 and a private score of 0.36278 and achieved 2nd place in the Google Landmark Retrieval Competition 2020.
Type theory in human-like learning and inference
Venue: arXiv cs.AI 2022
Year: 2022

Humans can generate reasonable answers to novel queries (Schulz, 2012): if I asked you what kind of food you want to eat for lunch, you would respond with a food, not a time. The thought that one would respond "After 4pm" to "What would you like to eat" is either a joke or a mistake, and seriously entertaining it as a lunch option would likely never happen in the first place. While understanding how people come up with new ideas, thoughts, explanations, and hypotheses that obey the basic constraints of a novel search space is of central importance to cognitive science, there is no agreed-on formal model for this kind of reasoning. We propose that a core component of any such reasoning system is a type theory: a formal imposition of structure on the kinds of computations an agent can perform, and how they're performed. We motivate this proposal with three empirical observations: adaptive constraints on learning and inference (i.e. generating reasonable hypotheses), how people draw distinctions between improbability and impossibility, and people's ability to reason about things at varying levels of abstraction.
Mining Duplicate Questions of Stack Overflow
Venue: arXiv cs.CL 2022
Year: 2022

There has a been a significant rise in the use of Community Question Answering sites (CQAs) over the last decade owing primarily to their ability to leverage the wisdom of the crowd. Duplicate questions have a crippling effect on the quality of these sites. Tackling duplicate questions is therefore an important step towards improving quality of CQAs. In this regard, we propose two neural network based architectures for duplicate question detection on Stack Overflow. We also propose explicitly modeling the code present in questions to achieve results that surpass the state of the art.
A Conditional-Probability-Distribution Model for Bandwidth Estimation with Application in Live Video Streaming
Venue: arXiv cs.MM 2022
Year: 2022

Experience of live video streaming can be improved if the video uploader has more accurate knowledge about the future available bandwidth. Because with such knowledge, one is able to know what sizes should he encode the frames to be in an ever-changing network. Researchers have developed some algorithms to predict throughputs in the literature, from where some are simple hence practical. However, limitation remains as most current bandwidth prediction methods are predicting a value, or a point estimate, of future bandwidth. Because in many practical scenarios, it is desirable to control the performance to some targets, e.g., video delivery rate over a given target percentage, which cannot be easily achieved via most current methods. In this work, we propose the use of probability distribution to model future bandwidth. Specifically, we model future bandwidth using past data transfer measurements and then derive a probability model for use in the application. This changes the selection of parameters in application into a probabilistic manner such that given target performance can be achieved in the long run. Inside our model, we use the conditional-probability method to correlate past and future bandwidth and hence further improve the estimating performance.
Putting Them under Microscope: A Fine-Grained Approach for Detecting Redundant Test Cases in Natural Language
Venue: arXiv cs.SE 2022
Year: 2022

Natural language (NL) documentation is the bridge between software managers and testers, and NL test cases are prevalent in system-level testing and other quality assurance activities. Due to reasons such as requirements redundancy, parallel testing, and tester turnover within long evolving history, there are inevitably lots of redundant test cases, which significantly increase the cost. Previous redundancy detection approaches typically treat the textual descriptions as a whole to compare their similarity and suffer from low precision. Our observation reveals that a test case can have explicit test-oriented entities, such as tested function Components, Constraints, etc; and there are also specific relations between these entities. This inspires us with a potential opportunity for accurate redundancy detection. In this paper, we first define five test-oriented entity categories and four associated relation categories and re-formulate the NL test case redundancy detection problem as the comparison of detailed testing content guided by the test-oriented entities and relations. Following that, we propose Tscope, a fine-grained approach for redundant NL test case detection by dissecting test cases into atomic test tuple(s) with the entities restricted by associated relations. To serve as the test case dissection, Tscope designs a context-aware model for the automatic entity and relation extraction. Evaluation on 3,467 test cases from ten projects shows Tscope could achieve 91.8% precision, 74.8% recall, and 82.4% F1, significantly outperforming state-of-the-art approaches and commonly-used classifiers. This new formulation of the NL test case redundant detection problem can motivate the follow-up studies to further improve this task and other related tasks involving NL descriptions.
Multiple Instance Learning for Detecting Anomalies over Sequential Real-World Datasets
Venue: arXiv cs.LG 2022
Year: 2022

Detecting anomalies over real-world datasets remains a challenging task. Data annotation is an intensive human labor problem, particularly in sequential datasets, where the start and end time of anomalies are not known. As a result, data collected from sequential real-world processes can be largely unlabeled or contain inaccurate labels. These characteristics challenge the application of anomaly detection techniques based on supervised learning. In contrast, Multiple Instance Learning (MIL) has been shown effective on problems with incomplete knowledge of labels in the training dataset, mainly due to the notion of bags. While largely under-leveraged for anomaly detection, MIL provides an appealing formulation for anomaly detection over real-world datasets, and it is the primary contribution of this paper. In this paper, we propose an MIL-based formulation and various algorithmic instantiations of this framework based on different design decisions for key components of the framework. We evaluate the resulting algorithms over four datasets that capture different physical processes along different modalities. The experimental evaluation draws out several observations. The MIL-based formulation performs no worse than single instance learning on easy to moderate datasets and outperforms single-instance learning on more challenging datasets. Altogether, the results show that the framework generalizes well over diverse datasets resulting from different real-world application domains.