OpenRaiser/Intern-Atlas huggingface dataset:
Alignment and stability of embeddings: measurement and inference improvement
Venue: arXiv cs.LG 2021
Year: 2021

Representation learning (RL) methods learn objects' latent embeddings where information is preserved by distances. Since distances are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. In the literature, embedding alignment has not been defined formally, explored theoretically, or analyzed empirically. Here, we explore the embedding alignment and its parts, provide the first formal definitions, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and by 40% in dynamic RL methods.
Formal Verification of a Fail-Operational Automotive Driving System
Venue: arXiv cs.SE 2021
Year: 2021

A fail-operational system for highly automated driving must complete the driving task even in the presence of a failure. This requires redundant architectures and a mechanism to reconfigure the system in case of a failure. Therefore, an arbitration logic is used. For functional safety, the switch-over to a fall-back level must be conducted in the presence of any electric and electronic failure. To provide evidence for a safety argumentation in compliance with ISO 26262, verification of the arbitration logic is necessary. The verification process provides confirmation of the correct failure reactions and that no unintended system states are attainable. Conventional safety analyses, such as the failure mode and effect analysis, have its limits in this regard. We present an analytical approach based on formal verification, in particular model checking, to verify the fail-operational behaviour of a driving system. For that reason, we model the system behaviour and the relevant architecture and formally specify the safety requirements. The scope of the analysis is defined according to the requirements of ISO 26262. We verify a fail-operational arbitration logic for highly automated driving in compliance with the industry standard. Our results show that formal methods for safety evaluation in automotive fail-operational driving systems can be successfully applied. We were able to detect failures, which would have been overlooked by other analyses and thus contribute to the development of safety critical functions.
GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery
Venue: arXiv cs.HC 2021
Year: 2021

People spend an enormous amount of time and effort looking for lost objects. To help remind people of the location of lost objects, various computational systems that provide information on their locations have been developed. However, prior systems for assisting people in finding objects require users to register the target objects in advance. This requirement imposes a cumbersome burden on the users, and the system cannot help remind them of unexpectedly lost objects. We propose GO-Finder ("Generic Object Finder"), a registration-free wearable camera based system for assisting people in finding an arbitrary number of objects based on two key features: automatic discovery of hand-held objects and image-based candidate selection. Given a video taken from a wearable camera, Go-Finder automatically detects and groups hand-held objects to form a visual timeline of the objects. Users can retrieve the last appearance of the object by browsing the timeline through a smartphone app. We conducted a user study to investigate how users benefit from using GO-Finder and confirmed improved accuracy and reduced mental load regarding the object search task by providing clear visual cues on object locations.
MIMOSA: Reducing Malware Analysis Overhead with Coverings
Venue: arXiv cs.CR 2021
Year: 2021

There is a growing body of malware samples that evade automated analysis and detection tools. Malware may measure fingerprints ("artifacts") of the underlying analysis tool or environment and change their behavior when artifacts are detected. While analysis tools can mitigate artifacts to reduce exposure, such concealment is expensive. However, not every sample checks for every type of artifact-analysis efficiency can be improved by mitigating only those artifacts most likely to be used by a sample. Using that insight, we propose MIMOSA, a system that identifies a small set of "covering" tool configurations that collectively defeat most malware samples with increased efficiency. MIMOSA identifies a set of tool configurations that maximize analysis throughput and detection accuracy while minimizing manual effort, enabling scalable automation to analyze stealthy malware. We evaluate our approach against a benchmark of 1535 labeled stealthy malware samples. Our approach increases analysis throughput over state of the art on over 95% of these samples. We also investigate cost-benefit tradeoffs between the fraction of successfully-analyzed samples and computing resources required. MIMOSA provides a practical, tunable method for efficiently deploying analysis resources.
Dynamic Longest Increasing Subsequence and the Erdös-Szekeres Partitioning Problem
Venue: arXiv cs.DS 2021
Year: 2021

In this paper, we provide new approximation algorithms for dynamic variations of the longest increasing subsequence (\textsf{LIS}) problem, and the complementary distance to monotonicity (\textsf{DTM}) problem. In this setting, operations of the following form arrive sequentially: (i) add an element, (ii) remove an element, or (iii) substitute an element for another. At every point in time, the algorithm has an approximation to the longest increasing subsequence (or distance to monotonicity). We present a $(1+ε)$-approximation algorithm for \textsf{DTM} with polylogarithmic worst-case update time and a constant factor approximation algorithm for \textsf{LIS} with worst-case update time $\tilde O(n^ε)$ for any constant $ε> 0$.% $n$ in the runtime denotes the size of the array at the time the operation arrives. Our dynamic algorithm for \textsf{LIS} leads to an almost optimal algorithm for the Erdös-Szekeres partitioning problem. Erdös-Szekeres partitioning problem was introduced by Erdös and Szekeres in 1935 and was known to be solvable in time $O(n^{1.5}\log n)$. Subsequent work improve the runtime to $O(n^{1.5})$ only in 1998. Our dynamic \textsf{LIS} algorithm leads to a solution for Erdös-Szekeres partitioning problem with runtime $\tilde O_ε(n^{1+ε})$ for any constant $ε> 0$.
Text line extraction using fully convolutional network and energy minimization
Venue: arXiv cs.CV 2021
Year: 2021

Text lines are important parts of handwritten document images and easier to analyze by further applications. Despite recent progress in text line detection, text line extraction from a handwritten document remains an unsolved task. This paper proposes to use a fully convolutional network for text line detection and energy minimization for text line extraction. Detected text lines are represented by blob lines that strike through the text lines. These blob lines assist an energy function for text line extraction. The detection stage can locate arbitrarily oriented text lines. Furthermore, the extraction stage is capable of finding out the pixels of text lines with various heights and interline proximity independent of their orientations. Besides, it can finely split the touching and overlapping text lines without an orientation assumption. We evaluate the proposed method on VML-AHTE, VML-MOC, and Diva-HisDB datasets. The VML-AHTE dataset contains overlapping, touching and close text lines with rich diacritics. The VML-MOC dataset is very challenging by its multiply oriented and skewed text lines. The Diva-HisDB dataset exhibits distinct text line heights and touching text lines. The results demonstrate the effectiveness of the method despite various types of challenges, yet using the same parameters in all the experiments.
Galaxy Image Translation with Semi-supervised Noise-reconstructed Generative Adversarial Networks
Venue: arXiv cs.CV 2021
Year: 2021

Image-to-image translation with Deep Learning neural networks, particularly with Generative Adversarial Networks (GANs), is one of the most powerful methods for simulating astronomical images. However, current work is limited to utilizing paired images with supervised translation, and there has been rare discussion on reconstructing noise background that encodes instrumental and observational effects. These limitations might be harmful for subsequent scientific applications in astrophysics. Therefore, we aim to develop methods for using unpaired images and preserving noise characteristics in image translation. In this work, we propose a two-way image translation model using GANs that exploits both paired and unpaired images in a semi-supervised manner, and introduce a noise emulating module that is able to learn and reconstruct noise characterized by high-frequency features. By experimenting on multi-band galaxy images from the Sloan Digital Sky Survey (SDSS) and the Canada France Hawaii Telescope Legacy Survey (CFHT), we show that our method recovers global and local properties effectively and outperforms benchmark image translation models. To our best knowledge, this work is the first attempt to apply semi-supervised methods and noise reconstruction techniques in astrophysical studies.
A Surrogate-Assisted Variable Grouping Algorithm for General Large Scale Global Optimization Problems
Venue: arXiv cs.NE 2021
Year: 2021

Problem decomposition plays a vital role when applying cooperative coevolution (CC) to large scale global optimization problems. However, most learning-based decomposition algorithms either only apply to additively separable problems or face the issue of false separability detections. Directing against these limitations, this study proposes a novel decomposition algorithm called surrogate-assisted variable grouping (SVG). SVG first designs a general-separability-oriented detection criterion according to whether the optimum of a variable changes with other variables. This criterion is consistent with the separability definition and thus endows SVG with broad applicability and high accuracy. To reduce the fitness evaluation requirement, SVG seeks the optimum of a variable with the help of a surrogate model rather than the original expensive high-dimensional model. Moreover, it converts the variable grouping process into a dynamic-binary-tree search one, which facilitates reutilizing historical separability detection information and thus reducing detection times. To evaluate the performance of SVG, a suite of benchmark functions with up to 2000 dimensions, including additively and non-additively separable ones, were designed. Experimental results on these functions indicate that, compared with six state-of-the-art decomposition algorithms, SVG possesses broader applicability and competitive efficiency. Furthermore, it can significantly enhance the optimization performance of CC.
Deep Reinforcement Learning for Producing Furniture Layout in Indoor Scenes
Venue: arXiv cs.CV 2021
Year: 2021

In the industrial interior design process, professional designers plan the size and position of furniture in a room to achieve a satisfactory design for selling. In this paper, we explore the interior scene design task as a Markov decision process (MDP), which is solved by deep reinforcement learning. The goal is to produce an accurate position and size of the furniture simultaneously for the indoor layout task. In particular, we first formulate the furniture layout task as a MDP problem by defining the state, action, and reward function. We then design the simulated environment and train reinforcement learning agents to produce the optimal layout for the MDP formulation. We conduct our experiments on a large-scale real-world interior layout dataset that contains industrial designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art model. The developed simulator and codes are available at \url{https://github.com/CODE-SUBMIT/simulator1}.
Hidden Markov Model-Based Encoding for Time-Correlated IoT Sources
Venue: arXiv cs.NI 2021
Year: 2021

As the use of Internet of Things (IoT) devices for monitoring purposes becomes ubiquitous, the efficiency of sensor communication is a major issue for the modern Internet. Channel coding is less efficient for extremely short packets, and traditional techniques that rely on source compression require extensive signaling or pre-existing knowledge of the source dynamics. In this work, we propose an encoding and decoding scheme that learns source dynamics online using a Hidden Markov Model (HMM), puncturing a short packet code to outperform existing compression-based approaches. Our approach shows significant performance improvements for sources that are highly correlated in time, with no additional complexity on the sender side.
Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets
Venue: arXiv cs.CV 2021
Year: 2021

This work studies Hyperspectral image (HSI) super-resolution (SR). HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples. This work addresses these issues with three contributions. First, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. Second, we propose a simple, yet effective data augmentation routine, termed Spectral Mixup, to construct effective virtual training samples to enlarge the training set. Finally, we extend the network to a semi-supervised setting so that it can learn from datasets containing only low-resolution HSIs. With these contributions, our method is able to learn from heterogeneous datasets and lift the requirement for having a large amount of HD HSI training samples. Extensive experiments on four standard datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. Code has been made available at https://github.com/kli8996/HSISR.
Interpretable Models for Granger Causality Using Self-explaining Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Exploratory analysis of time series data can yield a better understanding of complex dynamical systems. Granger causality is a practical framework for analysing interactions in sequential data, applied in a wide range of domains. In this paper, we propose a novel framework for inferring multivariate Granger causality under nonlinear dynamics based on an extension of self-explaining neural networks. This framework is more interpretable than other neural-network-based techniques for inferring Granger causality, since in addition to relational inference, it also allows detecting signs of Granger-causal effects and inspecting their variability over time. In comprehensive experiments on simulated data, we show that our framework performs on par with several powerful baseline methods at inferring Granger causality and that it achieves better performance at inferring interaction signs. The results suggest that our framework is a viable and more interpretable alternative to sparse-input neural networks for inferring Granger causality.
Chronological Citation Recommendation with Time Preference
Venue: arXiv cs.IR 2021
Year: 2021

Citation recommendation is an important task to assist scholars in finding candidate literature to cite. Traditional studies focus on static models of recommending citations, which do not explicitly distinguish differences between papers that are caused by temporal variations. Although, some researchers have investigated chronological citation recommendation by adding time related function or modeling textual topics dynamically. These solutions can hardly cope with function generalization or cold-start problems when there is no information for user profiling or there are isolated papers never being cited. With the rise and fall of science paradigms, scientific topics tend to change and evolve over time. People would have the time preference when citing papers, since most of the theoretical basis exist in classical readings that published in old time, while new techniques are proposed in more recent papers. To explore chronological citation recommendation, this paper wants to predict the time preference based on user queries, which is a probability distribution of citing papers published in different time slices. Then, we use this time preference to re-rank the initial citation list obtained by content-based filtering. Experimental results demonstrate that task performance can be further enhanced by time preference and it's flexible to be added in other citation recommendation frameworks.
Human Action Recognition Based on Multi-scale Feature Maps from Depth Video Sequences
Venue: arXiv cs.CV 2021
Year: 2021

Human action recognition is an active research area in computer vision. Although great process has been made, previous methods mostly recognize actions based on depth data at only one scale, and thus they often neglect multi-scale features that provide additional information action recognition in practical application scenarios. In this paper, we present a novel framework focusing on multi-scale motion information to recognize human actions from depth video sequences. We propose a multi-scale feature map called Laplacian pyramid depth motion images(LP-DMI). We employ depth motion images (DMI) as the templates to generate the multi-scale static representation of actions. Then, we caculate LP-DMI to enhance multi-scale dynamic information of motions and reduces redundant static information in human bodies. We further extract the multi-granularity descriptor called LP-DMI-HOG to provide more discriminative features. Finally, we utilize extreme learning machine (ELM) for action classification. The proposed method yeilds the recognition accuracy of 93.41%, 85.12%, 91.94% on public MSRAction3D dataset, UTD-MHAD and DHA dataset. Through extensive experiments, we prove that our method outperforms state-of-the-art benchmarks.
Cooperative NOMA-Based User Pairing for URLLC: A Max-Min Fairness Approach
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, cooperative non-orthogonal multiple access (C-NOMA) is considered in short packet communications with finite blocklength (FBL) codes. The performance of a decode-and-forward (DF) relaying along with selection combining (SC) and maximum ratio combining (MRC) strategies at the receiver side is examined. We explore joint user pairing and resource allocation to maximize fair throughput in a downlink (DL) scenario. In each pair, the user with a stronger channel (strong user) acts as a relay for the other one (weak user), and optimal power and blocklength are allocated to achieve max-min throughput. To this end, first, only one pair is considered, and optimal resource allocation is explored. Also, a suboptimal algorithm is suggested, which converges to a near-optimal solution. Finally, the problem is extended to a general scenario, and a suboptimal C-NOMA-based user pairing is proposed. Numerical results show that the proposed C-NOMA scheme in both SC and MRC strategies significantly improves the users' fair throughput compared to the NOMA and OMA. It is also investigated that the proposed pairing scheme based on C-NOMA outperforms the Hybrid NOMA/OMA scheme from the average throughput perspective, while the fairness index degrades slightly.
Proof Automation in the Theory of Finite Sets and Finite Set Relation Algebra
Venue: arXiv cs.LO 2021
Year: 2021

{log} ('setlog') is a satisfiability solver for formulas of the theory of finite sets and finite set relation algebra (FSTRA). As such, it can be used as an automated theorem prover (ATP) for this theory. {log} is able to automatically prove a number of FSTRA theorems, but not all of them. Nevertheless, we have observed that many theorems that {log} cannot automatically prove can be divided into a few subgoals automatically dischargeable by {log}. The purpose of this work is to present a prototype interactive theorem prover (ITP), called {log}-ITP, providing evidence that a proper integration of {log} into world-class ITP's can deliver a great deal of proof automation concerning FSTRA. An empirical evaluation based on 210 theorems from the TPTP and Coq's SSReflect libraries shows a noticeable reduction in the size and complexity of the proofs with respect to Coq.
Deep Feedback Inverse Problem Solver
Venue: arXiv cs.CV 2021
Year: 2021

We present an efficient, effective, and generic approach towards solving inverse problems. The key idea is to leverage the feedback signal provided by the forward process and learn an iterative update model. Specifically, at each iteration, the neural network takes the feedback as input and outputs an update on the current estimation. Our approach does not have any restrictions on the forward process; it does not require any prior knowledge either. Through the feedback information, our model not only can produce accurate estimations that are coherent to the input observation but also is capable of recovering from early incorrect predictions. We verify the performance of our approach over a wide range of inverse problems, including 6-DOF pose estimation, illumination estimation, as well as inverse kinematics. Comparing to traditional optimization-based methods, we can achieve comparable or better performance while being two to three orders of magnitude faster. Compared to deep learning-based approaches, our model consistently improves the performance on all metrics. Please refer to the project page for videos, animations, supplementary materials, etc.
Metadata Analysis of Open Educational Resources
Venue: arXiv cs.CY 2021
Year: 2021

Open Educational Resources (OERs) are openly licensed educational materials that are widely used for learning. Nowadays, many online learning repositories provide millions of OERs. Therefore, it is exceedingly difficult for learners to find the most appropriate OER among these resources. Subsequently, the precise OER metadata is critical for providing high-quality services such as search and recommendation. Moreover, metadata facilitates the process of automatic OER quality control as the continuously increasing number of OERs makes manual quality control extremely difficult. This work uses the metadata of 8,887 OERs to perform an exploratory data analysis on OER metadata. Accordingly, this work proposes metadata-based scoring and prediction models to anticipate the quality of OERs. Based on the results, our analysis demonstrated that OER metadata and OER content qualities are closely related, as we could detect high-quality OERs with an accuracy of 94.6%. Our model was also evaluated on 884 educational videos from Youtube to show its applicability on other educational repositories.