OpenRaiser/Intern-Atlas huggingface dataset:
Enhanced Low-Complexity FDD System Feedback with Variable Bit Lengths via Generative Modeling
Venue: arXiv cs.IT 2023
Year: 2023

Recently, a versatile limited feedback scheme based on a Gaussian mixture model (GMM) was proposed for frequency division duplex (FDD) systems. This scheme provides high flexibility regarding various system parameters and is applicable to both point-to-point multiple-input multiple-output (MIMO) and multi-user MIMO (MU-MIMO) communications. The GMM is learned to cover the operation of all mobile terminals (MTs) located inside the base station (BS) cell, and each MT only needs to evaluate its strongest mixture component as feedback, eliminating the need for channel estimation at the MT. In this work, we extend the GMM-based feedback scheme to variable feedback lengths by leveraging a single learned GMM through merging or pruning of dispensable mixture components. Additionally, the GMM covariances are restricted to Toeplitz or circulant structure through model-based insights. These extensions significantly reduce the offloading amount and enhance the clustering ability of the GMM which, in turn, leads to an improved system performance. Simulation results for both point-to-point and multi-user systems demonstrate the effectiveness of the proposed extensions.
Supporting single responsibility through automated extract method refactoring
Venue: arXiv cs.SE 2023
Year: 2023

The responsibility of a method/function is to perform some desired computations and disseminate the results to its caller through various deliverables, including object fields and variables in output instructions. Based on this definition of responsibility, this paper offers a new algorithm to refactor long methods to those with a single responsibility. We propose a backward slicing algorithm to decompose a long method into slightly overlapping slices. The slices are computed for each output instruction, representing the outcome of a responsibility delegated to the method. The slices will be non-overlapping if the slicing criteria address the same output variable. The slices are further extracted as independent methods, invoked by the original method if certain behavioral preservations are made. The proposed method has been evaluated on the GEMS extract method refactoring benchmark and three real-world projects. On average, our experiments demonstrate at least a 29.6% improvement in precision and a 12.1% improvement in the recall of uncovering refactoring opportunities compared to the state-of-the-art approaches. Furthermore, our tool improves method-level cohesion metrics by an average of 20% after refactoring. Experimental results confirm the applicability of the proposed approach in extracting methods with a single responsibility.
Finding the Right Way to Rome: Effect-oriented Graph Transformation
Venue: arXiv cs.LO 2023
Year: 2023

Many applications of graph transformation require rules that change a graph without introducing new consistency violations. When designing such rules, it is natural to think about the desired outcome state, i.e., the desired effect, rather than the specific steps required to achieve it; these steps may vary depending on the specific rule-application context. Existing graph-transformation approaches either require a separate rule to be written for every possible application context or lack the ability to constrain the maximal change that a rule will create. We introduce effect-oriented graph transformation, shifting the semantics of a rule from specifying actions to representing the desired effect. A single effect-oriented rule can encode a large number of induced classic rules. Which of the potential actions is executed depends on the application context; ultimately, all ways lead to Rome. If a graph element to be deleted (created) by a potential action is already absent (present), this action need not be performed because the desired outcome is already present. We formally define effect-oriented graph transformation, show how matches can be computed without explicitly enumerating all induced classic rules, and report on a prototypical implementation of effect-oriented graph transformation in Henshin.
Tight Bounds for Chordal/Interval Vertex Deletion Parameterized by Treewidth
Venue: arXiv cs.DS 2023
Year: 2023

In Chordal/Interval Vertex Deletion we ask how many vertices one needs to remove from a graph to make it chordal (respectively: interval). We study these problems under the parameterization by treewidth $tw$ of the input graph $G$. On the one hand, we present an algorithm for Chordal Vertex Deletion with running time $2^{O(tw)} \cdot |V(G)|$, improving upon the running time $2^{O(tw^2)} \cdot |V(G)|^{O(1)}$ by Jansen, de Kroon, and Wlodarczyk (STOC'21). When a tree decomposition of width $tw$ is given, then the base of the exponent equals $2^{ω-1}\cdot 3 + 1$. Our algorithm is based on a novel link between chordal graphs and graphic matroids, which allows us to employ the framework of representative families. On the other hand, we prove that the known $2^{O(tw \log tw)} \cdot |V(G)|$-time algorithm for Interval Vertex Deletion cannot be improved assuming Exponential Time Hypothesis.
Repair of Reed-Solomon Codes in the Presence of Erroneous Nodes
Venue: arXiv cs.IT 2023
Year: 2023

We consider the repair scheme of Guruswami-Wootters for the Reed-Solomon code and ask: can we correctly repair a failed node in the presence of erroneous nodes? Equivalently, we consider the collection of downloaded traces as a code and investigate its code-distance properties. We propose three lower bounds on its minimum distance and study methods to efficiently correct errors close to these bounds.
Few-shot Domain-Adaptive Visually-fused Event Detection from Text
Venue: arXiv cs.CL 2023
Year: 2023

Incorporating auxiliary modalities such as images into event detection models has attracted increasing interest over the last few years. The complexity of natural language in describing situations has motivated researchers to leverage the related visual context to improve event detection performance. However, current approaches in this area suffer from data scarcity, where a large amount of labelled text-image pairs are required for model training. Furthermore, limited access to the visual context at inference time negatively impacts the performance of such models, which makes them practically ineffective in real-world scenarios. In this paper, we present a novel domain-adaptive visually-fused event detection approach that can be trained on a few labelled image-text paired data points. Specifically, we introduce a visual imaginator method that synthesises images from text in the absence of visual context. Moreover, the imaginator can be customised to a specific domain. In doing so, our model can leverage the capabilities of pre-trained vision-language models and can be trained in a few-shot setting. This also allows for effective inference where only single-modality data (i.e. text) is available. The experimental evaluation on the benchmark M2E2 dataset shows that our model outperforms existing state-of-the-art models, by up to 11 points.
Completeness Thresholds for Memory Safety of Array Traversing Programs
Venue: arXiv cs.LO 2023
Year: 2023

We report on intermediate results of -- to the best of our knowledge -- the first study of completeness thresholds for (partially) bounded memory safety proofs. Specifically, we consider heap-manipulating programs that iterate over arrays without allocating or freeing memory. In this setting, we present the first notion of completeness thresholds for program verification which reduce unbounded memory safety proofs to (partially) bounded ones. Moreover, we demonstrate that we can characterise completeness thresholds for simple classes of array traversing programs. Finally, we suggest avenues of research to scale this technique theoretically, i.e., to larger classes of programs (heap manipulation, tree-like data structures), and practically by highlighting automation opportunities.
Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network
Venue: arXiv cs.CV 2023
Year: 2023

In the context of robotic grasping, object segmentation encounters several difficulties when faced with dynamic conditions such as real-time operation, occlusion, low lighting, motion blur, and object size variability. In response to these challenges, we propose the Graph Mixer Neural Network that includes a novel collaborative contextual mixing layer, applied to 3D event graphs formed on asynchronous events. The proposed layer is designed to spread spatiotemporal correlation within an event graph at four nearest neighbor levels parallelly. We evaluate the effectiveness of our proposed method on the Event-based Segmentation (ESD) Dataset, which includes five unique image degradation challenges, including occlusion, blur, brightness, trajectory, scale variance, and segmentation of known and unknown objects. The results show that our proposed approach outperforms state-of-the-art methods in terms of mean intersection over the union and pixel accuracy. Code available at: https://github.com/sanket0707/GNN-Mixer.git
Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models
Venue: arXiv cs.CL 2023
Year: 2023

We propose Retrieval Augmented Generation (RAG) as an approach for automated radiology report writing that leverages multimodally aligned embeddings from a contrastively pretrained vision language model for retrieval of relevant candidate radiology text for an input radiology image and a general domain generative model like OpenAI text-davinci-003, gpt-3.5-turbo and gpt-4 for report generation using the relevant radiology text retrieved. This approach keeps hallucinated generations under check and provides capabilities to generate report content in the format we desire leveraging the instruction following capabilities of these generative models. Our approach achieves better clinical metrics with a BERTScore of 0.2865 (Δ+ 25.88%) and Semb score of 0.4026 (Δ+ 6.31%). Our approach can be broadly relevant for different clinical settings as it allows to augment the automated radiology report generation process with content relevant for that setting while also having the ability to inject user intents and requirements in the prompts as part of the report generation process to modulate the content and format of the generated reports as applicable for that clinical setting.
Fault-Tolerant ST-Diameter Oracles
Venue: arXiv cs.DS 2023
Year: 2023

We study the problem of estimating the $ST$-diameter of a graph that is subject to a bounded number of edge failures. An $f$-edge fault-tolerant $ST$-diameter oracle ($f$-FDO-$ST$) is a data structure that preprocesses a given graph $G$, two sets of vertices $S,T$, and positive integer $f$. When queried with a set $F$ of at most $f$ edges, the oracle returns an estimate $\widehat{D}$ of the $ST$-diameter $\operatorname{diam}(G-F,S,T)$, the maximum distance between vertices in $S$ and $T$ in $G-F$. The oracle has stretch $σ\geq 1$ if $\operatorname{diam}(G-F,S,T) \leq \widehat{D} \leq σ\operatorname{diam}(G-F,S,T)$. If $S$ and $T$ both contain all vertices, the data structure is called an $f$-edge fault-tolerant diameter oracle ($f$-FDO). An $f$-edge fault-tolerant distance sensitivity oracles ($f$-DSO) estimates the pairwise graph distances under up to $f$ failures. We design new $f$-FDOs and $f$-FDO-$ST$s by reducing their construction to that of all-pairs and single-source $f$-DSOs. We obtain several new tradeoffs between the size of the data structure, stretch guarantee, query and preprocessing times for diameter oracles by combining our black-box reductions with known results from the literature. We also provide an information-theoretic lower bound on the space requirement of approximate $f$-FDOs. We show that there exists a family of graphs for which any $f$-FDO with sensitivity $f \ge 2$ and stretch less than $5/3$ requires $Ω(n^{3/2})$ bits of space, regardless of the query time.
Governance of the AI, by the AI, and for the AI
Venue: arXiv cs.CY 2023
Year: 2023

Over the past half century, there have been several false dawns during which the "arrival" of world-changing artificial intelligence (AI) has been heralded. Tempting fate, the authors believe the age of AI has, indeed, finally arrived. Powerful image generators, such as DALL-E2 and Midjourney have suddenly allowed anyone with access the ability easily to create rich and complex art. In a similar vein, text generators, such as GPT3.5 (including ChatGPT) and BLOOM, allow users to compose detailed written descriptions of many topics of interest. And, it is even possible now for a person without extensive expertise in writing software to use AI to generate code capable of myriad applications. While AI will continue to evolve and improve, probably at a rapid rate, the current state of AI is already ushering in profound changes to many different sectors of society. Every new technology challenges the ability of humanity to govern it wisely. However, governance is usually viewed as both possible and necessary due to the disruption new technology often poses to social structures, industries, the environment, and other important human concerns. In this article, we offer an analysis of a range of interactions between AI and governance, with the hope that wise decisions may be made that maximize benefits and minimize costs. The article addresses two main aspects of this relationship: the governance of AI by humanity, and the governance of humanity by AI. The approach we have taken is itself informed by AI, as this article was written collaboratively by the authors and ChatGPT.
ChatGPT and Works Scholarly: Best Practices and Legal Pitfalls in Writing with AI
Venue: arXiv cs.CY 2023
Year: 2023

Recent advances in artificial intelligence (AI) have raised questions about whether the use of AI is appropriate and legal in various professional contexts. Here, we present a perspective on how scholars may approach writing in conjunction with AI, and offer approaches to evaluating whether or not such AI-writing violates copyright or falls within the safe harbor of fair use. We present a set of best practices for standard of care with regard to plagiarism, copyright, and fair use. As AI is likely to grow more capable in the coming years, it is appropriate to begin integrating AI into scholarly writing activities. We offer a framework for establishing sound legal and scholarly foundations.
AmGCL: Feature Imputation of Attribute Missing Graph via Self-supervised Contrastive Learning
Venue: arXiv cs.LG 2023
Year: 2023

Attribute graphs are ubiquitous in multimedia applications, and graph representation learning (GRL) has been successful in analyzing attribute graph data. However, incomplete graph data and missing node attributes can have a negative impact on media knowledge discovery. Existing methods for handling attribute missing graph have limited assumptions or fail to capture complex attribute-graph dependencies. To address these challenges, we propose Attribute missing Graph Contrastive Learning (AmGCL), a framework for handling missing node attributes in attribute graph data. AmGCL leverages Dirichlet energy minimization-based feature precoding to encode in missing attributes and a self-supervised Graph Augmentation Contrastive Learning Structure (GACLS) to learn latent variables from the encoded-in data. Specifically, AmGCL utilizies feature reconstruction based on structure-attribute energy minimization while maximizes the lower bound of evidence for latent representation mutual information. Our experimental results on multiple real-world datasets demonstrate that AmGCL outperforms state-of-the-art methods in both feature imputation and node classification tasks, indicating the effectiveness of our proposed method in real-world attribute graph analysis tasks.
Harnessing the Power of BERT in the Turkish Clinical Domain: Pretraining Approaches for Limited Data Scenarios
Venue: arXiv cs.CL 2023
Year: 2023

In recent years, major advancements in natural language processing (NLP) have been driven by the emergence of large language models (LLMs), which have significantly revolutionized research and development within the field. Building upon this progress, our study delves into the effects of various pre-training methodologies on Turkish clinical language models' performance in a multi-label classification task involving radiology reports, with a focus on addressing the challenges posed by limited language resources. Additionally, we evaluated the simultaneous pretraining approach by utilizing limited clinical task data for the first time. We developed four models, including TurkRadBERT-task v1, TurkRadBERT-task v2, TurkRadBERT-sim v1, and TurkRadBERT-sim v2. Our findings indicate that the general Turkish BERT model (BERTurk) and TurkRadBERT-task v1, both of which utilize knowledge from a substantial general-domain corpus, demonstrate the best overall performance. Although the task-adaptive pre-training approach has the potential to capture domain-specific patterns, it is constrained by the limited task-specific corpus and may be susceptible to overfitting. Furthermore, our results underscore the significance of domain-specific vocabulary during pre-training for enhancing model performance. Ultimately, we observe that the combination of general-domain knowledge and task-specific fine-tuning is essential for achieving optimal performance across a range of categories. This study offers valuable insights for developing effective Turkish clinical language models and can guide future research on pre-training techniques for other low-resource languages within the clinical domain.
RECIPE: Rateless Erasure Codes Induced by Protocol-Based Encoding
Venue: arXiv cs.IT 2023
Year: 2023

LT (Luby transform) codes are a celebrated family of rateless erasure codes (RECs). Most of existing LT codes were designed for applications in which a centralized encoder possesses all message blocks and is solely responsible for encoding them into codewords. Distributed LT codes, in which message blocks are physically scattered across multiple different locations (encoders) that need to collaboratively perform the encoding, has never been systemically studied before despite its growing importance in applications. In this work, we present the first systemic study of LT codes in the distributed setting, and make the following three major contributions. First, we show that only a proper subset of LT codes are feasible in the distributed setting, and give the sufficient and necessary condition for such feasibility. Second, we propose a distributed encoding protocol that can efficiently implement any feasible code. The protocol is parameterized by a so-called action probability array (APA) that is only a few KBs in size, and any feasible code corresponds to a valid APA setting and vice versa. Third, we propose two heuristic search algorithms that have led to the discovery of feasible codes that are much more efficient than the state of the art.
Improving Image-Based Precision Medicine with Uncertainty-Aware Causal Models
Venue: arXiv cs.LG 2023
Year: 2023

Image-based precision medicine aims to personalize treatment decisions based on an individual's unique imaging features so as to improve their clinical outcome. Machine learning frameworks that integrate uncertainty estimation as part of their treatment recommendations would be safer and more reliable. However, little work has been done in adapting uncertainty estimation techniques and validation metrics for precision medicine. In this paper, we use Bayesian deep learning for estimating the posterior distribution over factual and counterfactual outcomes on several treatments. This allows for estimating the uncertainty for each treatment option and for the individual treatment effects (ITE) between any two treatments. We train and evaluate this model to predict future new and enlarging T2 lesion counts on a large, multi-center dataset of MR brain images of patients with multiple sclerosis, exposed to several treatments during randomized controlled trials. We evaluate the correlation of the uncertainty estimate with the factual error, and, given the lack of ground truth counterfactual outcomes, demonstrate how uncertainty for the ITE prediction relates to bounds on the ITE error. Lastly, we demonstrate how knowledge of uncertainty could modify clinical decision-making to improve individual patient and clinical trial outcomes.
CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling approaches for NER
Venue: arXiv cs.CL 2023
Year: 2023

This paper summarizes the CLaC submission for the MultiCoNER 2 task which concerns the recognition of complex, fine-grained named entities. We compare two popular approaches for NER, namely Sequence Labeling and Span Prediction. We find that our best Span Prediction system performs slightly better than our best Sequence Labeling system on test data. Moreover, we find that using the larger version of XLM RoBERTa significantly improves performance. Post-competition experiments show that Span Prediction and Sequence Labeling approaches improve when they use special input tokens (
 and 
) of XLM-RoBERTa. The code for training all models, preprocessing, and post-processing is available at https://github.com/harshshredding/semeval2023-multiconer-paper.
Data-Free Learning of Reduced-Order Kinematics
Venue: arXiv cs.GR 2023
Year: 2023

Physical systems ranging from elastic bodies to kinematic linkages are defined on high-dimensional configuration spaces, yet their typical low-energy configurations are concentrated on much lower-dimensional subspaces. This work addresses the challenge of identifying such subspaces automatically: given as input an energy function for a high-dimensional system, we produce a low-dimensional map whose image parameterizes a diverse yet low-energy submanifold of configurations. The only additional input needed is a single seed configuration for the system to initialize our procedure; no dataset of trajectories is required. We represent subspaces as neural networks that map a low-dimensional latent vector to the full configuration space, and propose a training scheme to fit network parameters to any system of interest. This formulation is effective across a very general range of physical systems; our experiments demonstrate not only nonlinear and very low-dimensional elastic body and cloth subspaces, but also more general systems like colliding rigid bodies and linkages. We briefly explore applications built on this formulation, including manipulation, latent interpolation, and sampling.
NorBench -- A Benchmark for Norwegian Language Models
Venue: arXiv cs.CL 2023
Year: 2023

We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.