OpenRaiser/Intern-Atlas huggingface dataset:
LF-checker: Machine Learning Acceleration of Bounded Model Checking for Concurrency Verification (Competition Contribution)
Venue: arXiv cs.LG 2023
Year: 2023

We describe and evaluate LF-checker, a metaverifier tool based on machine learning. It extracts multiple features of the program under test and predicts the optimal configuration (flags) of a bounded model checker with a decision tree. Our current work is specialised in concurrency verification and employs ESBMC as a back-end verification engine. In the paper, we demonstrate that LF-checker achieves better results than the default configuration of the underlying verification engine.
Differential Spiral Joint Mechanism for Coupled Variable Stiffness Actuation
Venue: arXiv cs.RO 2023
Year: 2023

In this study, we present the Differential Spiral Joint (DSJ) mechanism for variable stiffness actuation in tendon-driven robots. The DSJ mechanism semi-decouples the modulation of position and mechanical stiffness, allowing independent trajectory tracking in different parameter space. Past studies show that increasing the mechanical stiffness achieves the wider range of renderable stiffness, whereas decreasing the mechanical stiffness improves the quality of actuator decoupling and shock absorbance. Therefore, it is often useful to modulate the mechanical stiffness to balance the required level of stiffness and safety. In addition, the DSJ mechanism offers a compact form factor, which is suitable for applications where the size and weight are important. The performance of the DSJ mechanism in various areas is validated through a set of experiments.
Ensemble Transfer Learning for Multilingual Coreference Resolution
Venue: arXiv cs.CL 2023
Year: 2023

Entity coreference resolution is an important research problem with many applications, including information extraction and question answering. Coreference resolution for English has been studied extensively. However, there is relatively little work for other languages. A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. To overcome this challenge, we design a simple but effective ensemble-based framework that combines various transfer learning (TL) techniques. We first train several models using different TL methods. Then, during inference, we compute the unweighted average scores of the models' predictions to extract the final set of predicted clusters. Furthermore, we also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts. Leveraging the idea that the coreferential links naturally exist between anchor texts pointing to the same article, our method builds a sizeable distantly-supervised dataset for the target language that consists of tens of thousands of documents. We can pre-train a model on the pseudo-labeled dataset before finetuning it on the final target dataset. Experimental results on two benchmark datasets, OntoNotes and SemEval, confirm the effectiveness of our methods. Our best ensembles consistently outperform the baseline approach of simple training by up to 7.68% in the F1 score. These ensembles also achieve new state-of-the-art results for three languages: Arabic, Dutch, and Spanish.
Improving Autonomous Vehicle Mapping and Navigation in Work Zones Using Crowdsourcing Vehicle Trajectories
Venue: arXiv cs.RO 2023
Year: 2023

Prevalent solutions for Connected and Autonomous vehicle (CAV) mapping include high definition map (HD map) or real-time Simultaneous Localization and Mapping (SLAM). Both methods only rely on vehicle itself (onboard sensors or embedded maps) and can not adapt well to temporarily changed drivable areas such as work zones. Navigating CAVs in such areas heavily relies on how the vehicle defines drivable areas based on perception information. Difficulties in improving perception accuracy and ensuring the correct interpretation of perception results are challenging to the vehicle in these situations. This paper presents a prototype that introduces crowdsourcing trajectories information into the mapping process to enhance CAV's understanding on the drivable area and traffic rules. A Gaussian Mixture Model (GMM) is applied to construct the temporarily changed drivable area and occupancy grid map (OGM) based on crowdsourcing trajectories. The proposed method is compared with SLAM without any human driving information. Our method has adapted well with the downstream path planning and vehicle control module, and the CAV did not violate driving rule, which a pure SLAM method did not achieve.
Deterministic Online Classification: Non-iteratively Reweighted Recursive Least-Squares for Binary Class Rebalancing
Venue: arXiv cs.LG 2023
Year: 2023

Deterministic solutions are becoming more critical for interpretability. Weighted Least-Squares (WLS) has been widely used as a deterministic batch solution with a specific weight design. In the online settings of WLS, exact reweighting is necessary to converge to its batch settings. In order to comply with its necessity, the iteratively reweighted least-squares algorithm is mainly utilized with a linearly growing time complexity which is not attractive for online learning. Due to the high and growing computational costs, an efficient online formulation of reweighted least-squares is desired. We introduce a new deterministic online classification algorithm of WLS with a constant time complexity for binary class rebalancing. We demonstrate that our proposed online formulation exactly converges to its batch formulation and outperforms existing state-of-the-art stochastic online binary classification algorithms in real-world data sets empirically.
Optimising complexity of CNN models for resource constrained devices: QRS detection case study
Venue: arXiv cs.LG 2023
Year: 2023

Traditional DL models are complex and resource hungry and thus, care needs to be taken in designing Internet of (medical) things (IoT, or IoMT) applications balancing efficiency-complexity trade-off. Recent IoT solutions tend to avoid using deep-learning methods due to such complexities, and rather classical filter-based methods are commonly used. We hypothesize that a shallow CNN model can offer satisfactory level of performance in combination by leveraging other essential solution-components, such as post-processing that is suitable for resource constrained environment. In an IoMT application context, QRS-detection and R-peak localisation from ECG signal as a case study, the complexities of CNN models and post-processing were varied to identify a set of combinations suitable for a range of target resource-limited environments. To the best of our knowledge, finding a deploy-able configuration, by incrementally increasing the CNN model complexity, as required to match the target's resource capacity, and leveraging the strength of post-processing, is the first of its kind. The results show that a shallow 2-layer CNN with a suitable post-processing can achieve $>$90\% F1-score, and the scores continue to improving for 8-32 layer CNNs, which can be used to profile target constraint environment. The outcome shows that it is possible to design an optimal DL solution with known target performance characteristics and resource (computing capacity, and memory) constraints.
Refutations of pebble minimization via output languages
Venue: arXiv cs.FL 2023
Year: 2023

Polyregular functions are the class of string-to-string functions definable by pebble transducers, an extension of finite-state automata with outputs and multiple two-way reading heads (pebbles) with a stack discipline. If a polyregular function can be computed with $k$ pebbles, then its output length is bounded by a polynomial of degree $k$ in the input length. But Bojańczyk has shown that the converse fails. In this paper, we provide two alternative easier proofs. The first establishes by elementary means that some quadratic polyregular function requires 3 pebbles. The second proof - just as short, albeit less elementary - shows a stronger statement: for every $k$, there exists some polyregular function with quadratic growth whose output language differs from that of any $k$-fold composition of macro tree transducers (and which therefore cannot be computed by a $k$-pebble transducer). Along the way, we also refute a conjectured logical characterization of polyblind functions.
Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks
Venue: arXiv cs.NE 2023
Year: 2023

Throughout history, the development of artificial intelligence, particularly artificial neural networks, has been open to and constantly inspired by the increasingly deepened understanding of the brain, such as the inspiration of neocognitron, which is the pioneering work of convolutional neural networks. Per the motives of the emerging field: NeuroAI, a great amount of neuroscience knowledge can help catalyze the next generation of AI by endowing a network with more powerful capabilities. As we know, the human brain has numerous morphologically and functionally different neurons, while artificial neural networks are almost exclusively built on a single neuron type. In the human brain, neuronal diversity is an enabling factor for all kinds of biological intelligent behaviors. Since an artificial network is a miniature of the human brain, introducing neuronal diversity should be valuable in terms of addressing those essential problems of artificial networks such as efficiency, interpretability, and memory. In this Primer, we first discuss the preliminaries of biological neuronal diversity and the characteristics of information transmission and processing in a biological neuron. Then, we review studies of designing new neurons for artificial networks. Next, we discuss what gains can neuronal diversity bring into artificial networks and exemplary applications in several important fields. Lastly, we discuss the challenges and future directions of neuronal diversity to explore the potential of NeuroAI.
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion
Venue: arXiv cs.LG 2023
Year: 2023

In federated learning (FL), the communication constraint between the remote learners and the Parameter Server (PS) is a crucial bottleneck. For this reason, model updates must be compressed so as to minimize the loss in accuracy resulting from the communication constraint. This paper proposes ``\emph{${\bf M}$-magnitude weighted $L_{\bf 2}$ distortion + $\bf 2$ degrees of freedom''} (M22) algorithm, a rate-distortion inspired approach to gradient compression for federated training of deep neural networks (DNNs). In particular, we propose a family of distortion measures between the original gradient and the reconstruction we referred to as ``$M$-magnitude weighted $L_2$'' distortion, and we assume that gradient updates follow an i.i.d. distribution -- generalized normal or Weibull, which have two degrees of freedom. In both the distortion measure and the gradient, there is one free parameter for each that can be fitted as a function of the iteration number. Given a choice of gradient distribution and distortion measure, we design the quantizer minimizing the expected distortion in gradient reconstruction. To measure the gradient compression performance under a communication constraint, we define the \emph{per-bit accuracy} as the optimal improvement in accuracy that one bit of communication brings to the centralized model over the training period. Using this performance measure, we systematically benchmark the choice of gradient distribution and distortion measure. We provide substantial insights on the role of these choices and argue that significant performance improvements can be attained using such a rate-distortion inspired compressor.
Approximating Knapsack and Partition via Dense Subset Sums
Venue: arXiv cs.DS 2023
Year: 2023

Knapsack and Partition are two important additive problems whose fine-grained complexities in the $(1-\varepsilon)$-approximation setting are not yet settled. In this work, we make progress on both problems by giving improved algorithms. - Knapsack can be $(1 - \varepsilon)$-approximated in $\tilde O(n + (1/\varepsilon) ^ {2.2} )$ time, improving the previous $\tilde O(n + (1/\varepsilon) ^ {2.25} )$ by Jin (ICALP'19). There is a known conditional lower bound of $(n+\varepsilon)^{2-o(1)}$ based on $(\min,+)$-convolution hypothesis. - Partition can be $(1 - \varepsilon)$-approximated in $\tilde O(n + (1/\varepsilon) ^ {1.25} )$ time, improving the previous $\tilde O(n + (1/\varepsilon) ^ {1.5} )$ by Bringmann and Nakos (SODA'21). There is a known conditional lower bound of $(1/\varepsilon)^{1-o(1)}$ based on Strong Exponential Time Hypothesis. Both of our new algorithms apply the additive combinatorial results on dense subset sums by Galil and Margalit (SICOMP'91), Bringmann and Wellnitz (SODA'21). Such techniques have not been explored in the context of Knapsack prior to our work. In addition, we design several new methods to speed up the divide-and-conquer steps which naturally arise in solving additive problems.
A New Dynamic Programming Approach for Spanning Trees with Chain Constraints and Beyond
Venue: arXiv cs.DS 2023
Year: 2023

Short spanning trees subject to additional constraints are important building blocks in various approximation algorithms. Especially in the context of the Traveling Salesman Problem (TSP), new techniques for finding spanning trees with well-defined properties have been crucial in recent progress. We consider the problem of finding a spanning tree subject to constraints on the edges in cuts forming a laminar family of small width. Our main contribution is a new dynamic programming approach where the value of a table entry does not only depend on the values of previous table entries, as it is usually the case, but also on a specific representative solution saved together with each table entry. This allows for handling a broad range of constraint types. In combination with other techniques -- including negatively correlated rounding and a polyhedral approach that, in the problems we consider, allows for avoiding potential losses in the objective through the randomized rounding -- we obtain several new results. We first present a quasi-polynomial time algorithm for the Minimum Chain-Constrained Spanning Tree Problem with an essentially optimal guarantee. More precisely, each chain constraint is violated by a factor of at most $1+\varepsilon$, and the cost is no larger than that of an optimal solution not violating any chain constraint. The best previous procedure is a bicriteria approximation violating each chain constraint by up to a constant factor and losing another factor in the objective. Moreover, our approach can naturally handle lower bounds on the chain constraints, and it can be extended to constraints on cuts forming a laminar family of constant width. Furthermore, we show how our approach can also handle parity constraints (or, more precisely, a proxy thereof) as used in the context of (Path) TSP and one of its generalizations, and discuss implications in this context.
Splitting Plane Graphs to Outerplanarity
Venue: arXiv cs.CG 2023
Year: 2023

Vertex splitting replaces a vertex by two copies and partitions its incident edges amongst the copies. This problem has been studied as a graph editing operation to achieve desired properties with as few splits as possible, most often planarity, for which the problem is NP-hard. Here we study how to minimize the number of splits to turn a plane graph into an outerplane one. We tackle this problem by establishing a direct connection between splitting a plane graph to outerplanarity, finding a connected face cover, and finding a feedback vertex set in its dual. We prove NP-completeness for plane biconnected graphs, while we show that a polynomial-time algorithm exists for maximal planar graphs. Finally, we provide upper and lower bounds for certain families of maximal planar graphs.
The Energy Worker Profiler from Technologies to Skills to Realize Energy Efficiency in Manufacturing
Venue: arXiv cs.CL 2023
Year: 2023

In recent years, the manufacturing sector has been responsible for nearly 55 percent of total energy consumption, inducing a major impact on the global ecosystem. Although stricter regulations, restrictions on heavy manufacturing and technological advances are increasing its sustainability, zero-emission and fuel-efficient manufacturing is still considered a utopian target. In parallel,companies that have invested in digital innovation now need to align their internal competencies to maximize their return on investment. Moreover, a primary feature of Industry 4.0 is the digitization of production processes, which offers the opportunity to optimize energy consumption. However, given the speed with which innovation manifests itself, tools capable of measuring the impact that technology is having on digital and green professions and skills are still being designed. In light of the above, in this article we present the Worker Profiler, a software designed to map the skills currently possessed by workers, identifying misalignment with those they should ideally possess to meet the renewed demands that digital innovation and environmental preservation impose. The creation of the Worker Profiler consists of two steps: first, the authors inferred the key technologies and skills for the area of interest, isolating those with markedly increasing patent trends and identifying green and digital enabling skills and occupations. Thus, the software was designed and implemented at the user-interface level. The output of the self-assessment is the definition of the missing digital and green skills and the job roles closest to the starting one in terms of current skills; both the results enable the definition of a customized retraining strategy. The tool has shown evidence of being user-friendly, effective in identifying skills gaps and easily adaptable to other contexts.
HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Images
Venue: arXiv cs.CV 2023
Year: 2023

Visual question answering (VQA) is an important and challenging multimodal task in computer vision. Recently, a few efforts have been made to bring VQA task to aerial images, due to its potential real-world applications in disaster monitoring, urban planning, and digital earth product generation. However, not only the huge variation in the appearance, scale and orientation of the concepts in aerial images, but also the scarcity of the well-annotated datasets restricts the development of VQA in this domain. In this paper, we introduce a new dataset, HRVQA, which provides collected 53512 aerial images of 1024*1024 pixels and semi-automatically generated 1070240 QA pairs. To benchmark the understanding capability of VQA models for aerial images, we evaluate the relevant methods on HRVQA. Moreover, we propose a novel model, GFTransformer, with gated attention modules and a mutual fusion module. The experiments show that the proposed dataset is quite challenging, especially the specific attribute related questions. Our method achieves superior performance in comparison to the previous state-of-the-art approaches. The dataset and the source code will be released at https://hrvqa.nl/.
ECGAN: Self-supervised generative adversarial network for electrocardiography
Venue: arXiv cs.LG 2023
Year: 2023

High-quality synthetic data can support the development of effective predictive models for biomedical tasks, especially in rare diseases or when subject to compelling privacy constraints. These limitations, for instance, negatively impact open access to electrocardiography datasets about arrhythmias. This work introduces a self-supervised approach to the generation of synthetic electrocardiography time series which is shown to promote morphological plausibility. Our model (ECGAN) allows conditioning the generative process for specific rhythm abnormalities, enhancing synchronization and diversity across samples with respect to literature models. A dedicated sample quality assessment framework is also defined, leveraging arrhythmia classifiers. The empirical results highlight a substantial improvement against state-of-the-art generative models for sequences and audio synthesis.
SpArX: Sparse Argumentative Explanations for Neural Networks [Technical Report]
Venue: arXiv cs.AI 2023
Year: 2023

Neural networks (NNs) have various applications in AI, but explaining their decisions remains challenging. Existing approaches often focus on explaining how changing individual inputs affects NNs' outputs. However, an explanation that is consistent with the input-output behaviour of an NN is not necessarily faithful to the actual mechanics thereof. In this paper, we exploit relationships between multi-layer perceptrons (MLPs) and quantitative argumentation frameworks (QAFs) to create argumentative explanations for the mechanics of MLPs. Our SpArX method first sparsifies the MLP while maintaining as much of the original structure as possible. It then translates the sparse MLP into an equivalent QAF to shed light on the underlying decision process of the MLP, producing global and/or local explanations. We demonstrate experimentally that SpArX can give more faithful explanations than existing approaches, while simultaneously providing deeper insights into the actual reasoning process of MLPs.
Concept of Delivery System in the Smart City Environment
Venue: arXiv cs.CY 2023
Year: 2023

Regarding to the smart city infrastructures, there is a demand for big data processing and its further usage. This data can be gained by various means. There are many IoT devices in the city, which can communicate and share the information about the environment which they are situated in. Moreover every personal mobile device can also participate in this process and help to gain data via various applications. Every app provides the useful data, enabling the location and data sharing. This data can be further processed and used for improving the city infrastructure, transport or other services. We designed the system for shared delivery process, which can help to achieve the described situation. It consists of frontend and backend part. The frontend part, multiplatform mobile app, represents the graphical interface and the backend part represents the database for the gained data.
Injecting the BM25 Score as Text Improves BERT-Based Re-rankers
Venue: arXiv cs.IR 2023
Year: 2023

In this paper we propose a novel approach for combining first-stage lexical retrieval models and Transformer-based re-rankers: we inject the relevance score of the lexical model as a token in the middle of the input of the cross-encoder re-ranker. It was shown in prior work that interpolation between the relevance score of lexical and BERT-based re-rankers may not consistently result in higher effectiveness. Our idea is motivated by the finding that BERT models can capture numeric information. We compare several representations of the BM25 score and inject them as text in the input of four different cross-encoders. We additionally analyze the effect for different query types, and investigate the effectiveness of our method for capturing exact matching relevance. Evaluation on the MSMARCO Passage collection and the TREC DL collections shows that the proposed method significantly improves over all cross-encoder re-rankers as well as the common interpolation methods. We show that the improvement is consistent for all query types. We also find an improvement in exact matching capabilities over both BM25 and the cross-encoders. Our findings indicate that cross-encoder re-rankers can efficiently be improved without additional computational burden and extra steps in the pipeline by explicitly adding the output of the first-stage ranker to the model input, and this effect is robust for different models and query types.