OpenRaiser/Intern-Atlas huggingface dataset:
Wiener Graph Deconvolutional Network Improves Graph Self-Supervised Learning
Venue: arXiv cs.LG 2022
Year: 2022

Graph self-supervised learning (SSL) has been vastly employed to learn representations from unlabeled graphs. Existing methods can be roughly divided into predictive learning and contrastive learning, where the latter one attracts more research attention with better empirical performance. We argue that, however, predictive models weaponed with powerful decoder could achieve comparable or even better representation power than contrastive models. In this work, we propose a Wiener Graph Deconvolutional Network (WGDN), an augmentation-adaptive decoder empowered by graph wiener filter to perform information reconstruction. Theoretical analysis proves the superior reconstruction ability of graph wiener filter. Extensive experimental results on various datasets demonstrate the effectiveness of our approach.
Don't Look Up: Ubiquitous Data Exfiltration Pathways in Commercial Spaces
Venue: arXiv cs.CR 2022
Year: 2022

We show that as a side effect of building code requirements, almost all commercial buildings today are vulnerable to a novel data exfiltration attack, even if they are air-gapped and secured against traditional attacks. The new attack uses vibrations from an inconspicuous transmitter to send data across the building's physical infrastructure to a receiver. Our analysis and experiments with several large real-world buildings show a single-frequency bit rate of 300Kbps, which is sufficient to transmit ordinary files, real-time MP3-quality audio, or periodic high-quality still photos. The attacker can use multiple channels to transmit, for example, real-time MP4-quality video. We discuss the difficulty of detecting the attack and the viability of various potential countermeasures.
VLCap: Vision-Language with Contrastive Learning for Coherent Video Paragraph Captioning
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we leverage the human perceiving process, that involves vision and language interaction, to generate a coherent paragraph description of untrimmed videos. We propose vision-language (VL) features consisting of two modalities, i.e., (i) vision modality to capture global visual content of the entire scene and (ii) language modality to extract scene elements description of both human and non-human objects (e.g. animals, vehicles, etc), visual and non-visual elements (e.g. relations, activities, etc). Furthermore, we propose to train our proposed VLCap under a contrastive learning VL loss. The experiments and ablation studies on ActivityNet Captions and YouCookII datasets show that our VLCap outperforms existing SOTA methods on both accuracy and diversity metrics.
Balancing Run-Length Straight-Line Programs*
Venue: arXiv cs.DS 2022
Year: 2022

It was recently proved that any SLP generating a given string $w$ can be transformed in linear time into an equivalent balanced SLP of the same asymptotic size. We show that this result also holds for RLSLPs, which are SLPs extended with run-length rules of the form $A \rightarrow B^t$ for $t>2$, deriving $\texttt{exp}(A) = \texttt{exp}(B)^t$. An immediate consequence is the simplification of the algorithm for extracting substrings of an RLSLP-compressed string. We also show that several problems like answering RMQs and computing Karp-Rabin fingerprints on substrings can be solved in $\mathcal{O}(g_{rl})$ space and $\mathcal{O}(\log n)$ time, $g_{rl}$ being the size of the smallest RLSLP generating the string, of length $n$. We extend the result to solving more general operations on string ranges, in $\mathcal{O}(g_{rl})$ space and $\mathcal{O}(\log n)$ applications of the operation. In general, the smallest RLSLP can be asymptotically smaller than the smallest SLP by up to an $\mathcal{O}(\log n)$ factor, so our results can make a difference in terms of the space needed for computing these operations efficiently for some string families.
De-END: Decoder-driven Watermarking Network
Venue: arXiv cs.MM 2022
Year: 2022

With recent advances in machine learning, researchers are now able to solve traditional problems with new solutions. In the area of digital watermarking, deep-learning-based watermarking technique is being extensively studied. Most existing approaches adopt a similar encoder-driven scheme which we name END (Encoder-NoiseLayer-Decoder) architecture. In this paper, we revamp the architecture and creatively design a decoder-driven watermarking network dubbed De-END which greatly outperforms the existing END-based methods. The motivation for designing De-END originated from the potential drawback we discovered in END architecture: The encoder may embed redundant features that are not necessary for decoding, limiting the performance of the whole network. We conducted a detailed analysis and found that such limitations are caused by unsatisfactory coupling between the encoder and decoder in END. De-END addresses such drawbacks by adopting a Decoder-Encoder-Noiselayer-Decoder architecture. In De-END, the host image is firstly processed by the decoder to generate a latent feature map instead of being directly fed into the encoder. This latent feature map is concatenated to the original watermark message and then processed by the encoder. This change in design is crucial as it makes the feature of encoder and decoder directly shared thus the encoder and decoder are better coupled. We conducted extensive experiments and the results show that this framework outperforms the existing state-of-the-art (SOTA) END-based deep learning watermarking both in visual quality and robustness. On the premise of the same decoder structure, the visual quality (measured by PSNR) of De-END improves by 1.6dB (45.16dB to 46.84dB), and extraction accuracy after JPEG compression (QF=50) distortion outperforms more than 4% (94.9% to 99.1%).
Beating Greedy Matching in Sublinear Time
Venue: arXiv cs.DS 2022
Year: 2022

We study sublinear time algorithms for estimating the size of maximum matching in graphs. Our main result is a $(\frac{1}{2}+Ω(1))$-approximation algorithm which can be implemented in $O(n^{1+ε})$ time, where $n$ is the number of vertices and the constant $ε> 0$ can be made arbitrarily small. The best known lower bound for the problem is $Ω(n)$, which holds for any constant approximation. Existing algorithms either obtain the greedy bound of $\frac{1}{2}$-approximation [Behnezhad FOCS'21], or require some assumption on the maximum degree to run in $o(n^2)$-time [Yoshida, Yamamoto, and Ito STOC'09]. We improve over these by designing a less "adaptive" augmentation algorithm for maximum matching that might be of independent interest.
Leveraging Language for Accelerated Learning of Tool Manipulation
Venue: arXiv cs.RO 2022
Year: 2022

Robust and generalized tool manipulation requires an understanding of the properties and affordances of different tools. We investigate whether linguistic information about a tool (e.g., its geometry, common uses) can help control policies adapt faster to new tools for a given task. We obtain diverse descriptions of various tools in natural language and use pre-trained language models to generate their feature representations. We then perform language-conditioned meta-learning to learn policies that can efficiently adapt to new tools given their corresponding text descriptions. Our results demonstrate that combining linguistic information and meta-learning significantly accelerates tool learning in several manipulation tasks including pushing, lifting, sweeping, and hammering.
Probabilistic network topology prediction for active planning:An adaptive algorithm and application
Venue: arXiv cs.RO 2022
Year: 2022

This paper tackles the problem of active planning to achieve cooperative localization for multi-robot systems (MRS) under measurement uncertainty in GNSS-limited scenarios. Specifically, we address the issue of accurately predicting the probability of a future connection between two robots equipped with range-based measurement devices. Due to the limited range of the equipped sensors, edges in the network connection topology will be created or destroyed as the robots move with respect to one another. Accurately predicting the future existence of an edge, given imperfect state estimation and noisy actuation, is therefore a challenging task. An adaptive power series expansion (or APSE) algorithm is developed based on current estimates and control candidates. Such an algorithm applies the power series expansion formula of the quadratic positive form in a normal distribution. Finite-term approximation is made to realize the computational tractability. Further analyses are presented to show that the truncation error in the finite-term approximation can be theoretically reduced to a desired threshold by adaptively choosing the summation degree of the power series. Several sufficient conditions are rigorously derived as the selection principles. Finally, extensive simulation results and comparisons, with respect to both single and multi-robot cases, validate that a formally computed and therefore more accurate probability of future topology can help improve the performance of active planning under uncertainty.
Kernel Attention Transformer (KAT) for Histopathology Whole Slide Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Transformer has been widely used in histopathology whole slide image (WSI) classification for the purpose of tumor grading, prognosis analysis, etc. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits the effectiveness and efficiency in the application to gigapixel histopathology images. In this paper, we propose a kernel attention Transformer (KAT) for histopathology WSI classification. The information transmission of the tokens is achieved by cross-attention between the tokens and a set of kernels related to a set of positional anchors on the WSI. Compared to the common Transformer structure, the proposed KAT can better describe the hierarchical context information of the local regions of the WSI and meanwhile maintains a lower computational complexity. The proposed method was evaluated on a gastric dataset with 2040 WSIs and an endometrial dataset with 2560 WSIs, and was compared with 6 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI classification and is superior to the state-of-the-art methods. The code is available at https://github.com/zhengyushan/kat.
Measuring and Improving the Use of Graph Information in Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) have been widely used for representation learning on graph data. However, there is limited understanding on how much performance GNNs actually gain from graph data. This paper introduces a context-surrounding GNN framework and proposes two smoothness metrics to measure the quantity and quality of information obtained from graph data. A new GNN model, called CS-GNN, is then designed to improve the use of graph information based on the smoothness values of a graph. CS-GNN is shown to achieve better performance than existing methods in different types of real graphs.
A Model-Based Approach for Specifying Changes in Replications of Empirical Studies in Computer Science
Venue: arXiv cs.SE 2022
Year: 2022

Context: The need of replicating empirical studies in Computer Science (CS) is widely recognized among the research community to consolidate acquired knowledge generalizing results. It is essential to report the changes of each replication to understand the evolution of the experimental validity across a family of studies. Unfortunately, the lack of proposals undermines these objectives. Objective. The main goal of our work is to provide researchers in CS, and in other areas of research, with a systematic, tool-supported approach for the reporting of changes in the replications of their empirical studies. Method: Applying DSR, we have developed and validated a composite artifact consisting of (i) a metamodel of the relevant concepts of replications and their changes; (ii) templates and linguistic patterns for reporting those concepts; and (iii) a proof-of-concept model-based software tool that supports the proposed approach. For its validation, we have carried out a multiple case study including 9 families of empirical studies from CS and Agrobiology. The 9 families encompass 23 replication studies and 92 replication changes, for which we have analyzed the suitability of our proposal. Results: The multiple case study revealed some initial limitations of our approach related to threats to experimental validity or context variables. After several improvement iterations, all the 92 replication changes could be properly specified, including also their qualitatively estimated effects on experimental validity across the family of experiments and its corresponding visualization. Conclusions: Our proposal for the specification of replication changes seems to fit the needs not only of replications in CS, but also in other research areas. Nevertheless, further research is needed to improve it and to disseminate its use among the research community.
ML-Based Approach for NFL Defensive Pass Interference Prediction Using GPS Tracking Data
Venue: arXiv cs.LG 2022
Year: 2022

Defensive Pass Interference (DPI) is one of the most impactful penalties in the NFL. DPI is a spot foul, yielding an automatic first down to the team in possession. With such an influence on the game, referees have no room for a mistake. It is also a very rare event, which happens 1-2 times per 100 pass attempts. With technology improving and many IoT wearables being put on the athletes to collect valuable data, there is a solid ground for applying machine learning (ML) techniques to improve every aspect of the game. The work presented here is the first attempt in predicting DPI using player tracking GPS data. The data we used was collected by NFL's Next Gen Stats throughout the 2018 regular season. We present ML models for highly imbalanced time-series binary classification: LSTM, GRU, ANN, and Multivariate LSTM-FCN. Results showed that using GPS tracking data to predict DPI has limited success. The best performing models had high recall with low precision which resulted in the classification of many false positive examples. Looking closely at the data confirmed that there is just not enough information to determine whether a foul was committed. This study might serve as a filter for multi-step pipeline for video sequence classification which could be able to solve this problem.
Consistency-preserving Visual Question Answering in Medical Imaging
Venue: arXiv cs.CV 2022
Year: 2022

Visual Question Answering (VQA) models take an image and a natural-language question as input and infer the answer to the question. Recently, VQA systems in medical imaging have gained popularity thanks to potential advantages such as patient engagement and second opinions for clinicians. While most research efforts have been focused on improving architectures and overcoming data-related limitations, answer consistency has been overlooked even though it plays a critical role in establishing trustworthy models. In this work, we propose a novel loss function and corresponding training procedure that allows the inclusion of relations between questions into the training process. Specifically, we consider the case where implications between perception and reasoning questions are known a-priori. To show the benefits of our approach, we evaluate it on the clinically relevant task of Diabetic Macular Edema (DME) staging from fundus imaging. Our experiments show that our method outperforms state-of-the-art baselines, not only by improving model consistency, but also in terms of overall model accuracy. Our code and data are available at https://github.com/sergiotasconmorales/consistency_vqa.
Deep Active Learning for Regression Using $ε$-weighted Hybrid Query Strategy
Venue: arXiv cs.LG 2022
Year: 2022

Designing an inexpensive approximate surrogate model that captures the salient features of an expensive high-fidelity behavior is a prevalent approach in design optimization. In recent times, Deep Learning (DL) models are being used as a promising surrogate computational model for engineering problems. However, the main challenge in creating a DL-based surrogate is to simulate/label a large number of design points, which is time-consuming for computationally costly and/or high-dimensional engineering problems. In the present work, we propose a novel sampling technique by combining the active learning (AL) method with DL. We call this method $ε$-weighted hybrid query strategy ($ε$-HQS) , which focuses on the evaluation of the surrogate at each learning iteration and provides an estimate of the failure probability of the surrogate in the Design Space. By reusing already collected training and test data, the learned failure probability guides the next iteration's sampling process to the region of the high probability of failure. During the empirical evaluation, better accuracy of the surrogate was observed in comparison to other methods of sample selection. We empirically evaluated this method in two different engineering design domains, finite element based static stress analysis of submarine pressure vessel(computationally costly process) and second submarine propeller design( high dimensional problem). https://github.com/vardhah/epsilon_weighted_Hybrid_Query_Strategy
PARTICUL: Part Identification with Confidence measure using Unsupervised Learning
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present PARTICUL, a novel algorithm for unsupervised learning of part detectors from datasets used in fine-grained recognition. It exploits the macro-similarities of all images in the training set in order to mine for recurring patterns in the feature space of a pre-trained convolutional neural network. We propose new objective functions enforcing the locality and unicity of the detected parts. Additionally, we embed our detectors with a confidence measure based on correlation scores, allowing the system to estimate the visibility of each part. We apply our method on two public fine-grained datasets (Caltech-UCSD Bird 200 and Stanford Cars) and show that our detectors can consistently highlight parts of the object while providing a good measure of the confidence in their prediction. We also demonstrate that these detectors can be directly used to build part-based fine-grained classifiers that provide a good compromise between the transparency of prototype-based approaches and the performance of non-interpretable methods.
Open Set Classification of Untranscribed Handwritten Documents
Venue: arXiv cs.CV 2022
Year: 2022

Huge amounts of digital page images of important manuscripts are preserved in archives worldwide. The amounts are so large that it is generally unfeasible for archivists to adequately tag most of the documents with the required metadata so as to low proper organization of the archives and effective exploration by scholars and the general public. The class or ``typology'' of a document is perhaps the most important tag to be included in the metadata. The technical problem is one of automatic classification of documents, each consisting of a set of untranscribed handwritten text images, by the textual contents of the images. The approach considered is based on ``probabilistic indexing'', a relatively novel technology which allows to effectively represent the intrinsic word-level uncertainty exhibited by handwritten text images. We assess the performance of this approach on a large collection of complex notarial manuscripts from the Spanish Archivo Hostórico Provincial de Cádiz, with promising results.
Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safety
Venue: arXiv cs.AI 2022
Year: 2022

AI systems have been widely adopted across various domains in the real world. However, in high-value, sensitive, or safety-critical applications such as self-management for personalized health or food recommendation with a specific purpose (e.g., allergy-aware recipe recommendations), their adoption is unlikely. Firstly, the AI system needs to follow guidelines or well-defined processes set by experts; the data alone will not be adequate. For example, to diagnose the severity of depression, mental healthcare providers use Patient Health Questionnaire (PHQ-9). So if an AI system were to be used for diagnosis, the medical guideline implied by the PHQ-9 needs to be used. Likewise, a nutritionist's knowledge and steps would need to be used for an AI system that guides a diabetic patient in developing a food plan. Second, the BlackBox nature typical of many current AI systems will not work; the user of an AI system will need to be able to give user-understandable explanations, explanations constructed using concepts that humans can understand and are familiar with. This is the key to eliciting confidence and trust in the AI system. For such applications, in addition to data and domain knowledge, the AI systems need to have access to and use the Process Knowledge, an ordered set of steps that the AI system needs to use or adhere to.
A Simple and Scalable Tensor Completion Algorithm via Latent Invariant Constraint for Recommendation System
Venue: arXiv cs.IR 2022
Year: 2022

In this paper we provide a latent-variable formulation and solution to the recommender system (RS) problem in terms of a fundamental property that any reasonable solution should be expected to satisfy. Specifically, we examine a novel tensor completion method to efficiently and accurately learn parameters of a model for the unobservable personal preferences that underly user ratings. By regularizing the tensor decomposition with a single latent invariant, we achieve three properties for a reliable recommender system: (1) uniqueness of the tensor completion result with minimal assumptions, (2) unit consistency that is independent of arbitrary preferences of users, and (3) a consensus ordering guarantee that provides consistent ranking between observed and unobserved rating scores. Our algorithm leads to a simple and elegant recommendation framework that has linear computational complexity and with no hyperparameter tuning. We provide empirical results demonstrating that the approach significantly outperforms current state-of-the-art methods.
Conceptual Modeling of Actions
Venue: arXiv cs.SE 2022
Year: 2022

Modeling in software engineering includes constructing static, dynamic, and behavioral representations. In describing system behavior, actions and states are two of the most commonly used concepts. In this paper, we focus on the notion of action. It generally held that the meaning of the concept of action is not that easy to grasp. According to some researchers, many existing systems do involve the notion of action, but in an obscure way. In Unified Modeling Language (UML), an action is a single atomic step within an activity, i.e., it is not further decomposed within the activity. Activity represents a behavior that is composed of actions. This paper contributes to the establishment of a broader interdisciplinary understanding of the notion of action in conceptual modeling based on a model called the thinging machine (TM). The TM uses only five primitive actions: create, process, release, transfer, and receive. The goal of such a venture is to improve the process of developing conceptual models by refining basic concepts such as action and event. To demonstrate how TM modeling represents actions, UML activity and Business Process Model and Notation (BPMN) diagrams are re-modeled in terms of the five TM actions. The results reveal the viability of the TM s five actions in modeling and relate them to other important notions such as activity, event, and behavior.