OpenRaiser/Intern-Atlas huggingface dataset:
Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing
Venue: arXiv cs.RO 2023
Year: 2023

From dishwashers to cabinets, humans interact with articulated objects every day, and for a robot to assist in common manipulation tasks, it must learn a representation of articulation. Recent deep learning learning methods can provide powerful vision-based priors on the affordance of articulated objects from previous, possibly simulated, experiences. In contrast, many works estimate articulation by observing the object in motion, requiring the robot to already be interacting with the object. In this work, we propose to use the best of both worlds by introducing an online estimation method that merges vision-based affordance predictions from a neural network with interactive kinematic sensing in an analytical model. Our work has the benefit of using vision to predict an articulation model before touching the object, while also being able to update the model quickly from kinematic sensing during the interaction. In this paper, we implement a full system using shared autonomy for robotic opening of articulated objects, in particular objects in which the articulation is not apparent from vision alone. We implemented our system on a real robot and performed several autonomous closed-loop experiments in which the robot had to open a door with unknown joint while estimating the articulation online. Our system achieved an 80% success rate for autonomous opening of unknown articulated objects.
Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs
Venue: arXiv cs.LG 2023
Year: 2023

Most knowledge graph completion (KGC) methods learn latent representations of entities and relations of a given graph by mapping them into a vector space. Although the majority of these methods focus on static knowledge graphs, a large number of publicly available KGs contain temporal information stating the time instant/period over which a certain fact has been true. Such graphs are often known as temporal knowledge graphs. Furthermore, knowledge graphs may also contain textual descriptions of entities and relations. Both temporal information and textual descriptions are not taken into account during representation learning by static KGC methods, and only structural information of the graph is leveraged. Recently, some studies have used temporal information to improve link prediction, yet they do not exploit textual descriptions and do not support inductive inference (prediction on entities that have not been seen in training). We propose a novel framework called TEMT that exploits the power of pre-trained language models (PLMs) for text-enhanced temporal knowledge graph completion. The knowledge stored in the parameters of a PLM allows TEMT to produce rich semantic representations of facts and to generalize on previously unseen entities. TEMT leverages textual and temporal information available in a KG, treats them separately, and fuses them to get plausibility scores of facts. Unlike previous approaches, TEMT effectively captures dependencies across different time points and enables predictions on unseen entities. To assess the performance of TEMT, we carried out several experiments including time interval prediction, both in transductive and inductive settings, and triple classification. The experimental results show that TEMT is competitive with the state-of-the-art.
Byzantine Resilient Computing with the Cloud
Venue: arXiv cs.DC 2023
Year: 2023

We study a framework for modeling distributed network systems assisted by a reliable and powerful cloud service. Our framework aims at capturing hybrid systems based on a point to point message passing network of machines, with the additional capability of being able to access the services of a trusted high-performance external entity (the cloud). We focus on one concrete aspect that was not studied before, namely, ways of utilizing the cloud assistance in order to attain increased resilience against Byzantine behavior of machines in the network. Our network is modeled as a congested clique comprising $k$ machines that are completely connected to form a clique and can communicate with each other by passing small messages. In every execution, up to $βk$ machines (for suitable values of $β\in [0, 1)$) are allowed to be Byzantine, i.e., behave maliciously including colluding with each other, with the remaining $γk$ or more machines being \emph{honest} (for $γ=1-β$). Additionally, the machines in our congested clique can access data through a trusted cloud via queries. This externality of the data captures many real-world distributed computing scenarios and provides a natural context for exploring Byzantine resilience for essentially all conceivable problems. Moreover, we are no longer bound by the usual limits of $β< 1/3$ or even $β< 1/2$ that are typically seen in Byzantine Agreement. We focus on a few fundamental problems. We start with the ${\textsf{Download}}$ problem, wherein the cloud stores $n$ bits and these $n$ bits must be downloaded to all of the $k$ machines. In addition to ${\textsf{Download}}$, we also consider the problem of computing the ${\textsf{Disjunction}}$ and ${\textsf{Parity}}$ of the bits in the cloud. We study these problems under several settings comprising various $β$ values and adversarial capabilities.
A Comprehensive Survey of Document-level Relation Extraction (2016-2023)
Venue: arXiv cs.CL 2023
Year: 2023

Document-level relation extraction (DocRE) is an active area of research in natural language processing (NLP) concerned with identifying and extracting relationships between entities beyond sentence boundaries. Compared to the more traditional sentence-level relation extraction, DocRE provides a broader context for analysis and is more challenging because it involves identifying relationships that may span multiple sentences or paragraphs. This task has gained increased interest as a viable solution to build and populate knowledge bases automatically from unstructured large-scale documents (e.g., scientific papers, legal contracts, or news articles), in order to have a better understanding of relationships between entities. This paper aims to provide a comprehensive overview of recent advances in this field, highlighting its different applications in comparison to sentence-level relation extraction.
gSPICE: Model-Based Event Shedding in Complex Event Processing
Venue: arXiv cs.DC 2023
Year: 2023

Overload situations, in the presence of resource limitations, in complex event processing (CEP) systems are typically handled using load shedding to maintain a given latency bound. However, load shedding might negatively impact the quality of results (QoR). To minimize the shedding impact on QoR, CEP researchers propose shedding approaches that drop events/internal state with the lowest importances/utilities. In both black-box and white-box shedding approaches, different features are used to predict these utilities. In this work, we propose a novel black-box shedding approach that uses a new set of features to drop events from the input event stream to maintain a given latency bound. Our approach uses a probabilistic model to predict these event utilities. Moreover, our approach uses Zobrist hashing and well-known machine learning models, e.g., decision trees and random forests, to handle the predicted event utilities. Through extensive evaluations on several synthetic and two real-world datasets and a representative set of CEP queries, we show that, in the majority of cases, our load shedding approach outperforms state-of-the-art black-box load shedding approaches, w.r.t. QoR.
Towards Novel Class Discovery: A Study in Novel Skin Lesions Clustering
Venue: arXiv cs.CV 2023
Year: 2023

Existing deep learning models have achieved promising performance in recognizing skin diseases from dermoscopic images. However, these models can only recognize samples from predefined categories, when they are deployed in the clinic, data from new unknown categories are constantly emerging. Therefore, it is crucial to automatically discover and identify new semantic categories from new data. In this paper, we propose a new novel class discovery framework for automatically discovering new semantic classes from dermoscopy image datasets based on the knowledge of known classes. Specifically, we first use contrastive learning to learn a robust and unbiased feature representation based on all data from known and unknown categories. We then propose an uncertainty-aware multi-view cross pseudo-supervision strategy, which is trained jointly on all categories of data using pseudo labels generated by a self-labeling strategy. Finally, we further refine the pseudo label by aggregating neighborhood information through local sample similarity to improve the clustering performance of the model for unknown categories. We conducted extensive experiments on the dermatology dataset ISIC 2019, and the experimental results show that our approach can effectively leverage knowledge from known categories to discover new semantic categories. We also further validated the effectiveness of the different modules through extensive ablation experiments. Our code will be released soon.
Innovation Modeling Grid
Venue: arXiv cs.SI 2023
Year: 2023

This technical document presents the committee driven innovation modeling methodology "Innovation Modeling Grid" in detail. This document is the successor of three publications on IMoG and focuses on presenting all details of the methodology
SIMD Everywhere Optimization from ARM NEON to RISC-V Vector Extensions
Venue: arXiv cs.DC 2023
Year: 2023

Many libraries, such as OpenCV, FFmpeg, XNNPACK, and Eigen, utilize Arm or x86 SIMD Intrinsics to optimize programs for performance. With the emergence of RISC-V Vector Extensions (RVV), there is a need to migrate these performance legacy codes for RVV. Currently, the migration of NEON code to RVV code requires manual rewriting, which is a time-consuming and error-prone process. In this work, we use the open source tool, "SIMD Everywhere" (SIMDe), to automate the migration. Our primary task is to enhance SIMDe to enable the conversion of ARM NEON Intrinsics types and functions to their corresponding RVV Intrinsics types and functions. For type conversion, we devise strategies to convert Neon Intrinsics types to RVV Intrinsics by considering the vector length agnostic (vla) architectures. With function conversions, we analyze commonly used conversion methods in SIMDe and develop customized conversions for each function based on the results of RVV code generations. In our experiments with Google XNNPACK library, our enhanced SIMDe achieves speedup ranging from 1.51x to 5.13x compared to the original SIMDe, which does not utilize customized RVV implementations for the conversions.
Navigating Healthcare Insights: A Birds Eye View of Explainability with Knowledge Graphs
Venue: arXiv cs.AI 2023
Year: 2023

Knowledge graphs (KGs) are gaining prominence in Healthcare AI, especially in drug discovery and pharmaceutical research as they provide a structured way to integrate diverse information sources, enhancing AI system interpretability. This interpretability is crucial in healthcare, where trust and transparency matter, and eXplainable AI (XAI) supports decision making for healthcare professionals. This overview summarizes recent literature on the impact of KGs in healthcare and their role in developing explainable AI models. We cover KG workflow, including construction, relationship extraction, reasoning, and their applications in areas like Drug-Drug Interactions (DDI), Drug Target Interactions (DTI), Drug Development (DD), Adverse Drug Reactions (ADR), and bioinformatics. We emphasize the importance of making KGs more interpretable through knowledge-infused learning in healthcare. Finally, we highlight research challenges and provide insights for future directions.
Novel Deep Learning Pipeline for Automatic Weapon Detection
Venue: arXiv cs.CV 2023
Year: 2023

Weapon and gun violence have recently become a pressing issue today. The degree of these crimes and activities has risen to the point of being termed as an epidemic. This prevalent misuse of weapons calls for an automatic system that detects weapons in real-time. Real-time surveillance video is captured and recorded in almost all public forums and places. These videos contain abundant raw data which can be extracted and processed into meaningful information. This paper proposes a novel pipeline consisting of an ensemble of convolutional neural networks with distinct architectures. Each neural network is trained with a unique mini-batch with little to no overlap in the training samples. This paper will present several promising results using multiple datasets associated with comparing the proposed architecture and state-of-the-art (SoA) models. The proposed pipeline produced an average increase of 5% in accuracy, specificity, and recall compared to the SoA systems.
ecoBLE: A Low-Computation Energy Consumption Prediction Framework for Bluetooth Low Energy
Venue: arXiv cs.NI 2023
Year: 2023

Bluetooth Low Energy (BLE) is a de-facto technology for Internet of Things (IoT) applications, promising very low energy consumption. However, this low energy consumption accounts only for the radio part, and it overlooks the energy consumption of other hardware and software components. Monitoring and predicting the energy consumption of IoT nodes after deployment can substantially aid in ensuring low energy consumption, calculating the remaining battery lifetime, predicting needed energy for energy-harvesting nodes, and detecting anomalies. In this paper, we introduce a Long Short-Term Memory Projection (LSTMP)-based BLE energy consumption prediction framework together with a dataset for a healthcare application scenario where BLE is widely adopted. Unlike radio-focused theoretical energy models, our framework provides a comprehensive energy consumption prediction, considering all components of the IoT node, including the radio, sensor as well as microcontroller unit (MCU). Our measurement-based results show that the proposed framework predicts the energy consumption of different BLE nodes with a Mean Absolute Percentage Error (MAPE) of up to 12%, giving comparable accuracy to state-of-the-art energy consumption prediction with a five times smaller prediction model size.
Small Teams Propel Fresh Ideas in Science and Technology
Venue: arXiv cs.DL 2023
Year: 2023

The past half-century has seen a dramatic increase in the scale and complexity of scientific research, to which researchers have responded by dedicating more time to education and training, narrowing their areas of specialization, and collaborating in larger teams. A widely held view is that such collaborations, by fostering specialization and encouraging novel combinations of ideas, accelerate scientific innovation. However, recent research challenges this notion, suggesting that small teams and solo researchers consistently disrupt science and technology with fresh ideas and opportunities, while larger teams tend to refine existing ones (Wu et al. 2019). This study, along with other relevant research, has garnered attention for challenging the zeitgeist of our time that views collaboration as the inevitable path forward in scientific and technological advancement. Yet, few studies have re-evaluated its central finding: the innovative advantage of small teams over large ones, using alternative measures. We explore innovation by identifying papers proposing new scientific concepts and patents introducing new technology codes. We analyzed 88 million research articles spanning from 1800 to 2020 and 7 million patent applications from 1976 to 2020 worldwide. Our findings confirm that while large teams contribute to development, small teams play a critical role in innovation by propelling fresh, original ideas in science and technology.
Efficient Training of One Class Classification-SVMs
Venue: arXiv cs.LG 2023
Year: 2023

This study examines the use of a highly effective training method to conduct one-class classification. The existence of both positive and negative examples in the training data is necessary to develop an effective classifier in common binary classification scenarios. Unfortunately, this criteria is not met in many domains. Here, there is just one class of examples. Classification algorithms that learn from solely positive input have been created to deal with this setting. In this paper, an effective algorithm for dual soft-margin one-class SVM training is presented. Our approach makes use of the Augmented Lagrangian (AL-FPGM), a variant of the Fast Projected Gradient Method. The FPGM requires only first derivatives, which for the dual soft margin OCC-SVM means computing mainly a matrix-vector product. Therefore, AL-FPGM, being computationally inexpensive, may complement existing quadratic programming solvers for training large SVMs. We extensively validate our approach over real-world datasets and demonstrate that our strategy obtains statistically significant results.
Neural scaling laws for phenotypic drug discovery
Venue: arXiv cs.LG 2023
Year: 2023

Recent breakthroughs by deep neural networks (DNNs) in natural language processing (NLP) and computer vision have been driven by a scale-up of models and data rather than the discovery of novel computing paradigms. Here, we investigate if scale can have a similar impact for models designed to aid small molecule drug discovery. We address this question through a large-scale and systematic analysis of how DNN size, data diet, and learning routines interact to impact accuracy on our Phenotypic Chemistry Arena (Pheno-CA) benchmark: a diverse set of drug development tasks posed on image-based high content screening data. Surprisingly, we find that DNNs explicitly supervised to solve tasks in the Pheno-CA do not continuously improve as their data and model size is scaled-up. To address this issue, we introduce a novel precursor task, the Inverse Biological Process (IBP), which is designed to resemble the causal objective functions that have proven successful for NLP. We indeed find that DNNs first trained with IBP then probed for performance on the Pheno-CA significantly outperform task-supervised DNNs. More importantly, the performance of these IBP-trained DNNs monotonically improves with data and model scale. Our findings reveal that the DNN ingredients needed to accurately solve small molecule drug development tasks are already in our hands, and project how much more experimental data is needed to achieve any desired level of improvement. We release our Pheno-CA benchmark and code to encourage further study of neural scaling laws for small molecule drug discovery.
Test-Case Quality -- Understanding Practitioners' Perspectives
Venue: arXiv cs.SE 2023
Year: 2023

Background: Test-case quality has always been one of the major concerns in software testing. To improve test-case quality, it is important to better understand how practitioners perceive the quality of test-cases. Objective: Motivated by that need, we investigated how practitioners define test-case quality and which aspects of test-cases are important for quality assessment. Method: We conducted semi-structured interviews with professional developers, testers and test architects from a multinational software company in Sweden. Before the interviews, we asked participants for actual test cases (written in natural language) that they perceive as good, normal, and bad respectively together with rationales for their assessment. We also compared their opinions on shared test cases and contrasted their views with the relevant literature. Results: We present a quality model which consists of 11 test-case quality attributes. We also identify a misalignment in defining test-case quality among practitioners and between academia and industry, along with suggestions for improving test-case quality in industry. Conclusion: The results show that practitioners' background, including roles and working experience, are critical dimensions of how test-case quality is defined and assessed.
PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformers
Venue: arXiv cs.LG 2023
Year: 2023

Approximating nonlinear differential equations using a neural network provides a robust and efficient tool for various scientific computing tasks, including real-time predictions, inverse problems, optimal controls, and surrogate modeling. Previous works have focused on embedding dynamical systems into networks through two approaches: learning a single solution operator (i.e., the mapping from input parametrized functions to solutions) or learning the governing system of equations (i.e., the constitutive model relative to the state variables). Both of these approaches yield different representations for the same underlying data or function. Additionally, observing that families of differential equations often share key characteristics, we seek one network representation across a wide range of equations. Our method, called Predicting Operators and Symbolic Expressions (PROSE), learns maps from multimodal inputs to multimodal outputs, capable of generating both numerical predictions and mathematical equations. By using a transformer structure and a feature fusion approach, our network can simultaneously embed sets of solution operators for various parametric differential equations using a single trained network. Detailed experiments demonstrate that the network benefits from its multimodal nature, resulting in improved prediction accuracy and better generalization. The network is shown to be able to handle noise in the data and errors in the symbolic representation, including noisy numerical values, model misspecification, and erroneous addition or deletion of terms. PROSE provides a new neural network framework for differential equations which allows for more flexibility and generality in learning operators and governing equations from data.
Optimal Nonlinearities Improve Generalization Performance of Random Features
Venue: arXiv cs.LG 2023
Year: 2023

Random feature model with a nonlinear activation function has been shown to perform asymptotically equivalent to a Gaussian model in terms of training and generalization errors. Analysis of the equivalent model reveals an important yet not fully understood role played by the activation function. To address this issue, we study the "parameters" of the equivalent model to achieve improved generalization performance for a given supervised learning problem. We show that acquired parameters from the Gaussian model enable us to define a set of optimal nonlinearities. We provide two example classes from this set, e.g., second-order polynomial and piecewise linear functions. These functions are optimized to improve generalization performance regardless of the actual form. We experiment with regression and classification problems, including synthetic and real (e.g., CIFAR10) data. Our numerical results validate that the optimized nonlinearities achieve better generalization performance than widely-used nonlinear functions such as ReLU. Furthermore, we illustrate that the proposed nonlinearities also mitigate the so-called double descent phenomenon, which is known as the non-monotonic generalization performance regarding the sample size and the model size.